Skip to main content
SpringerLink
Log in

Pre-denoising 3D Multi-scale Fusion Attention Network for Low-Light Enhancement

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Due to the underexposure, the lack of details and the noise issues, Low-light images always have a high degree of degradation. In this paper, we thoroughly study the degradation mechanism of low-light images and design a pre-denoising 3D multi-scale fusion attention network (P3DMFE) with Retinex decomposition theory. This work is divided into three modules, firstly, The proposed three-branch decomposition module decouples the original space into three sub-spaces: reflection decomposition, illumination decomposition and noise decomposition, where the noise decomposition allows us to obtain the higher-quality reflection map and illumination map. Secondly, the 3D multi-scale fusion improvement module removes the noise map and performs image reshaping, structure restoration and detail restoration on the combined reflection map and illumination map. Thirdly, the Illumination improvement module provides a suitable illumination map. The experimental results show that the proposed P3DMFE can not only enrich the details and improve the brightness and contrast of low-light images, but also have a good denoising effect. Specifically, the proposed method can achieve 22.04 PSNR, 0.84 SSIM, 1250.4 LOE and 5.03 NIQE on LOL dataset, which are the best performance compared with some state-of-the-art methods. The experiments on common low-light datasets such as NPE, VV, MIT5K, MEF, LIME, and DICM also verify the good generalization ability and superiority of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. Preprint arXiv:1409.0473

  2. Chen C, Chen Q, Xu J, Koltun V (2018) Learning to see in the dark. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3291–3300

  3. Chen J, Lei B, Song Q, Ying H, Chen DZ, Wu J (2020) A hierarchical graph network for 3d object detection on point clouds. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 392–401

  4. Chrzanowski K (2013) Review of night vision technology. Opto-Electron Rev 21(2):153–181

    Article  Google Scholar 

  5. Dong X, Wang G, Pang Y, Li W, Wen J, Meng W, Lu Y (2011) Fast efficient algorithm for enhancement of low lighting video. In: 2011 IEEE international conference on multimedia and expo, IEEE, pp 1–6

  6. Fu J, Zheng H, Mei T (2017) Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4438–4446

  7. Fu X, Zeng D, Huang Y, Ding X, Zhang XP (2013) A variational framework for single low light image enhancement using bright channel prior. In: 2013 IEEE global conference on signal and information processing, IEEE, pp 1085–1088

  8. Fu X, Zeng D, Huang Y, Liao Y, Ding X, Paisley J (2016) A fusion-based enhancing method for weakly illuminated images. Signal Process 129:82–96

    Article  Google Scholar 

  9. Fu X, Zeng D, Huang Y, Zhang XP, Ding X (2016) A weighted variational model for simultaneous reflectance and illumination estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2782–2790

  10. Gao G, Xu G, Yu Y, Xie J, Yang J, Yue D (2021) Mscfnet: a lightweight network with multi-scale context fusion for real-time semantic segmentation. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2021.3098355

    Article  Google Scholar 

  11. Gao G, Yang J, Jing XY, Shen F, Yang W, Yue D (2017) Learning robust and discriminative low-rank representations for face recognition with occlusion. Pattern Recognit 66:129–143

    Article  Google Scholar 

  12. Gonzalez RC, Woods RE (2008) Digital image processing, prentice hall. Upper Saddle River, NJ

    Google Scholar 

  13. Guo C, Li C, Guo J, Loy CC, Hou J, Kwong S, Cong R (2020) Zero-reference deep curve estimation for low-light image enhancement. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1780–1789

  14. Guo X, Li Y, Ling H (2016) Lime: low-light image enhancement via illumination map estimation. IEEE Trans Image Process 26(2):982–993

    Article  MathSciNet  MATH  Google Scholar 

  15. Han JH, Yang S, Lee BU (2010) A novel 3-d color histogram equalization method with uniform 1-d gray scale histogram. IEEE Trans Image Process 20(2):506–512

    Article  MathSciNet  MATH  Google Scholar 

  16. Hao S, Han X, Guo Y, Xu X, Wang M (2020) Low-light image enhancement with semi-decoupled decomposition. IEEE Trans Multimed 22(12):3025–3038

    Article  Google Scholar 

  17. Hu J, Shen L, Albanie S, Sun G, Vedaldi A (2018) Gather-excite: exploiting feature context in convolutional neural networks. Preprint arXiv:1810.12348

  18. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  19. Huang SC, Cheng FC, Chiu YS (2012) Efficient contrast enhancement using adaptive gamma correction with weighting distribution. IEEE Trans Image Process 22(3):1032–1041

    Article  MathSciNet  MATH  Google Scholar 

  20. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  21. Jiang Y, Gong X, Liu D, Cheng Y, Fang C, Shen X, Yang J, Zhou P, Wang Z (2021) Enlightengan: deep light enhancement without paired supervision. IEEE Trans Image Process 30:2340–2349

    Article  Google Scholar 

  22. Jobson DJ, Rahman ZU, Woodell GA (1997) A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans Image Process 6(7):965–976

    Article  Google Scholar 

  23. Jobson DJ, Rahman ZU, Woodell GA (1997) Properties and performance of a center/surround retinex. IEEE Trans Image Process 6(3):451–462

    Article  Google Scholar 

  24. Kim I, Baek W, Kim S (2020) Spatially attentive output layer for image classification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9533–9542

  25. Li J, Fang F, Mei K, Zhang G (2018) Multi-scale residual network for image super-resolution. In: Proceedings of the European conference on computer vision (ECCV), pp 517–532

  26. Li J, Feng X, Hua Z (2021) Low-light image enhancement via progressive-recursive network. IEEE Trans Circuits Syst Video Technol 31(11):4227–4240

    Article  Google Scholar 

  27. Li J, Li J, Fang F, Li F, Zhang G (2021) Luminance-aware pyramid network for low-light image enhancement. IEEE Trans Multim 23:3153–3165

  28. Li M, Liu J, Yang W, Sun X, Guo Z (2018) Structure-revealing low-light image enhancement via robust retinex model. IEEE Trans Image Process 27(6):2828–2841

    Article  MathSciNet  MATH  Google Scholar 

  29. Li X, Wang W, Hu X, Yang J (2019) Selective kernel networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 510–519

  30. Lim S, Kim W (2021) DSLR: Deep stacked Laplacian restorer for low-light image enhancement. IEEE Trans Multim 23:4272–4284

  31. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  32. Lore KG, Akintayo A, Sarkar S (2017) Llnet: a deep autoencoder approach to natural low-light image enhancement. Pattern Recogn 61:650–662

    Article  Google Scholar 

  33. Lu K, Zhang L (2021) TBEFN: A two-branch exposure-fusion network for low-light image enhancement. IEEE Trans Multim 23:4093–4105

  34. Lv F, Lu F, Wu J, Lim C (2018) Mbllen: Low-light image/video enhancement using CNNS. In: BMVC, p 220

  35. Ma T, Guo M, Yu Z, Chen Y, Ren X, Xi R, Li Y, Zhou X (2021) Retinexgan: unsupervised low-light enhancement with two-layer convolutional decomposition networks. IEEE Access 9:56539–56550

    Article  Google Scholar 

  36. McCahill M (2013) The surveillance web. Willan

  37. Mittal A, Soundararajan R, Bovik AC (2013) Making a completely blind image quality analyzer. IEEE Signal Process Lett 20(3):209–212

  38. Mnih V, Heess N, Graves A, et al (2014) Recurrent models of visual attention. In: Advances in neural information processing systems, pp 2204–2212

  39. Park J, Woo S, Lee JY, Kweon IS (2018) Bam: bottleneck attention module. Preprint arXiv:1807.06514

  40. Pisano ED, Zong S, Hemminger BM, DeLuca M, Johnston RE, Muller K, Braeuning MP, Pizer SM (1998) Contrast limited adaptive histogram equalization image processing to improve the detection of simulated spiculations in dense mammograms. J Digit Imaging 11(4):193

    Article  Google Scholar 

  41. Ren X, Li M, Cheng WH, Liu J (2018) Joint enhancement and denoising method via sequential decomposition. In: 2018 IEEE international symposium on circuits and systems (ISCAS), IEEE, pp 1–5

  42. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, . Springer, pp 234–241

  43. Roy AG, Navab N, Wachinger C (2018) Concurrent spatial and channel ‘squeeze & excitation’ in fully convolutional networks. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 421–429

  44. Shen L, Yue Z, Feng F, Chen Q, Liu S, Ma J (2017) Msr-net: low-light image enhancement using deep convolutional network. Preprint arXiv:1711.02488

  45. Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Wang X, Tang X (2017) Residual attention network for image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3156–3164

  46. Wang R, Zhang Q, Fu CW, Shen X, Zheng WS, Jia J (2019) Underexposed photo enhancement using deep illumination estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6849–6857

  47. Wang S, Zheng J, Hu HM, Li B (2013) Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans Image Process 22(9):3538–3548

    Article  Google Scholar 

  48. Wang W, Ng MK (2013) A variational histogram equalization method for image contrast enhancement. SIAM J Imag Sci 6(3):1823–1849

    Article  MathSciNet  MATH  Google Scholar 

  49. Wang W, Wei C, Yang W, Liu J (2018) Gladnet: Low-light enhancement network with global awareness. In: 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018), IEEE, pp 751–755

  50. Wei C, Wang W, Yang W, Liu J (2018) Deep retinex decomposition for low-light enhancement. Preprint arXiv:1808.04560

  51. Xu H, Zhai G, Wu X, Yang X (2013) Generalized equalization model for image enhancement. IEEE Trans Multim 16(1):68–82

    Article  Google Scholar 

  52. Xu K, Ba J, Kiros R, Cho K, Courville A, Salakhudinov R, Zemel R, Bengio Y (2015) Show, attend and tell: Neural image caption generation with visual attention. In: International conference on machine learning, PMLR, pp 2048–2057

  53. Xu K, Yang X, Yin B, Lau RW (2020) Learning to restore low-light images via decomposition-and-enhancement. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2281–2290

  54. Yang W, Wang S, Fang Y, Wang Y, Liu J (2020) From fidelity to perceptual quality: a semi-supervised approach for low-light image enhancement. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3063–3072

  55. Ying Z, Li G, Gao W A (2017) bio-inspired multi-exposure fusion framework for low-light image enhancement. Preprint arXiv:1711.00591

  56. Ying Z, Li G, Ren Y, Wang R, Wang W (2017) A new low-light image enhancement algorithm using camera response model. In: Proceedings of the IEEE international conference on computer vision workshops, pp 3015–3022

  57. Yurtsever E, Lambert J, Carballo A, Takeda K (2020) A survey of autonomous driving: common practices and emerging technologies. IEEE access 8:58443–58469

    Article  Google Scholar 

  58. Zhang L, Zhang L, Liu X, Shen Y, Zhang S, Zhao S (2019) Zero-shot restoration of back-lit images using deep internal learning. In: Proceedings of the 27th ACM international conference on multimedia, pp 1623–1631

  59. Zhang Y, Zhang J, Guo X (2019) Kindling the darkness: a practical low-light image enhancer. In: Proceedings of the 27th ACM international conference on multimedia, pp 1632–1640

  60. Zhu G, Ma L, Liu R, Fan X, Luo Z (2021) Collaborative reflectance-and-illumination learning for high-efficient low-light image enhancement. In: 2021 IEEE international conference on multimedia and expo (ICME), IEEE, pp 1–6

Download references

Acknowledgements

This study was funded by the Natural Science Foundation of Liaoning Province (No. 2020–MS–080), the National Natural Science Foundation of China (No. 61772125)

Author information

Authors and Affiliations

  1. College of Sciences, Northeastern University, Shenyang, 110819, China

    Hegui Zhu, Ziwei Zhang, Luyang Wang, Tian Geng & Xiangde Zhang

Authors
  1. Hegui Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ziwei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luyang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tian Geng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiangde Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hegui Zhu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, H., Zhang, Z., Wang, L. et al. Pre-denoising 3D Multi-scale Fusion Attention Network for Low-Light Enhancement. Neural Process Lett 55, 5717–5743 (2023). https://doi.org/10.1007/s11063-022-11107-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-11107-x

Keywords

Search

Navigation