Skip to main content
SpringerLink
Log in

Effective low-light image enhancement with multiscale and context learning network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Convolutional Neural Network (CNN) has been widely used in low-light image enhancement task, and has achieved good enhancement results. However, the enhancement results not only are limited by the convolution kernel, but also are affected by the different shapes and sizes of low-light regions. CNN can only capture local dependencies. It is difficult to obtain long-distance dependencies and multiscale features from images, resulting in over/under enhancement. To alleviate these problems, we propose a Multiscale and Context Learning Network (MCLNet) for adaptive low-light enhancement by multiscale feature extraction and global relationships learning. Concretely, in order to obtain discriminative representation in diverse low-light regions, we design an Attentive Residual Multiscale Block (ARMB) to captures valuable multiscale features through spatial attention mechanisms at different scales. Further, we propose a Bottleblock of Scale Aggregation Module (BSAM) to learn hierarchical discriminative features based on th ARBM. Finally, to further adaptive enhancement from globel view, we present a Context Encoding Module (CEM) to model long-distance dependencies by Transformer. Experimental results show that our proposed MCLNet achieves superior performance of low-light images enhancement than some state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability

The dataset analyzed in the current study is available from the corresponding author upon reasonable request. The models and code analyzed in the current study will soon be publicly available at https://github.com/hlinqiao/MCLNet.git.

Notes

  1. https://github.com/hlinqiao/MCLNet.git

  2. https://dragon.larc.nasa.gov/retinex/pao/news/

References

  1. Cai Z, Zhang Y, Manzi M, Oztireli C, Gross M, Aydin TO (2021) Robust image denoising using kernel predicting networks

  2. Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S (2020) End-to-end object detection with transformers. In: European conference on computer vision (ECCV). Springer, pp 213–229

  3. Chongyi L, Guo C, Loy CC (2021) Learning to enhance low-light image via zero-reference deep curve estimation. arXiv:2103.00860

  4. Dong X, Wang G, Pang Y, Li W, Wen J, Meng W, Lu Y (2011) Fast efficient algorithm for enhancement of low lighting video. In: 2011 IEEE international conference on multimedia and expo (ICME), pp 1–6

  5. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S et al (2020) An image is worth 16x16 words: transformers for image recognition at scale. arXiv:2010.11929

  6. Fu X, Zeng D, Huang Y, Zhang X-P, Ding X (2016) A weighted variational model for simultaneous reflectance and illumination estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2782–2790

  7. Fu Q, Di X, Zhang Y (2020) Learning an adaptive model for extreme low-light raw image processing. arXiv:2004.10447

  8. Ghosh S, Chaudhury KN (2019) Fast bright-pass bilateral filtering for low-light enhancement. In: 2019 IEEE international conference on image processing (ICIP). IEEE, pp 205–209

  9. Guo X, Li Y, Ling H (2016) Lime: low-light image enhancement via illumination map estimation. IEEE Trans Image Process 26(2):982–993

    Article  MathSciNet  MATH  Google Scholar 

  10. Guo C, Li C, Guo J, Loy CC, Hou J, Kwong S, Cong R (2020) Zero-reference deep curve estimation for low-light image enhancement. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1780–1789

  11. Huang S-C, Cheng F-C, Chiu Y-S (2012) Efficient contrast enhancement using adaptive gamma correction with weighting distribution. IEEE Trans on Image Process 22(3):1032–1041

    Article  MathSciNet  MATH  Google Scholar 

  12. Jiang Y, Chang S, Wang Z (2021) Transgan: two transformers can make one strong gan. 1(3) arXiv:2102.07074

  13. Jiang Y, Gong X, Liu D, Cheng Y, Fang C, Shen X, Yang J, Zhou P, Wang Z (2021) Enlightengan: deep light enhancement without paired supervision. IEEE Trans Image Process 30:2340–2349

    Article  Google Scholar 

  14. Jobson DJ, Rahman Z-U, Woodell GA (1997) A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans on Image Process 6(7):965–976

    Article  Google Scholar 

  15. Jobson DJ, Rahman Z-U, Woodell GA (1997) Properties and performance of a center/surround retinex. IEEE Trans Image Process 6(3):451–462

    Article  Google Scholar 

  16. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: European conference on computer vision (ECCV). Springer, pp 694–711

  17. Land EH (1964) The retinex. Am Sci 52(2):247–264

    Google Scholar 

  18. Li J, Fang F, Mei K, Zhang G (2018) Multi-scale residual network for image super-resolution. In: Proceedings of the european conference on computer vision (ECCV), pp 517–532

  19. Li M, Liu J, Yang W, Sun X, Guo Z (2018) Structure-revealing low-light image enhancement via robust retinex model. IEEE Trans Image Process 27(6):2828–2841

    Article  MathSciNet  MATH  Google Scholar 

  20. Li X, Guo X, Mei L, Shang M, Gao J, Shu M, Wang X (2020) Visual perception model for rapid and adaptive low-light image enhancement. arXiv:2005.07343

  21. Liu C, Sui X, Liu Y, Kuang X, Li G, Chen Q (2019) Adaptive contrast enhancement based on histogram modification framework. J Mod Opt 66(15):1590–1601

    Article  Google Scholar 

  22. Lore KG, Akintayo A, Sarkar S (2017) Llnet: a deep autoencoder approach to natural low-light image enhancement. Pattern Recogn 61:650–662

    Article  Google Scholar 

  23. Lv F, Lu F, Wu J, Lim C (2018) Mbllen: low-light image/video enhancement using cnns. In: British machine vision association (BMVC), p 220

  24. Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Tran on Image Process 21(12):4695–4708

    Article  MathSciNet  MATH  Google Scholar 

  25. Mittal A, Soundararajan R, Bovik AC (2012) Making a “completely blind” image quality analyzer. IEEE Sig Process Lett 20(3):209–212

    Article  Google Scholar 

  26. Mun J, Jang Y, Nam Y, Kim J (2019) Edge-enhancing bi-histogram equalisation using guided image filter. J Vis Commun Image Represent 58:688–700

    Article  Google Scholar 

  27. Rahman Z-U, Jobson DJ, Woodell GA (1996) Multi-scale retinex for color image enhancement. In: Proceedings of 3rd IEEE international conference on image processing, vol 3. IEEE, pp 1003–1006

  28. Ren X, Li M, Cheng W-H, Liu J (2018) Joint enhancement and denoising method via sequential decomposition. In: 2018 IEEE international symposium on circuits and systems (ISCAS). IEEE, pp 1–5

  29. Shen L, Yue Z, Feng F, Chen Q, Liu S, Ma J (2017) Msr-net: low-light image enhancement using deep convolutional network. arXiv:1711.02488

  30. Srinivas A, Lin T-Y, Parmar N, Shlens J, Abbeel P, Vaswani A (2021) Bottleneck transformers for visual recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 16519–16529

  31. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-first AAAI conference on artificial intelligence

  32. Tan SF, Isa NAM (2019) Exposure based multi-histogram equalization contrast enhancement for non-uniform illumination images. IEEE Access 7:70842–70861

    Article  Google Scholar 

  33. Tao L, Zhu C, Xiang G, Li Y, Jia H, Xie X (2017) Llcnn: a convolutional neural network for low-light image enhancement. In: 2017 IEEE visual communications and image processing (VCIP). IEEE, pp 1–4

  34. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  35. Wang Y, Chen Q, Zhang B (1999) Image enhancement based on equal area dualistic sub-image histogram equalization method. IEEE Trans Consum Electron 45(1):68–75

    Article  Google Scholar 

  36. Wang Z, Simoncelli EP, Bovik AC (2003) Multiscale structural similarity for image quality assessment. In: The thrity-seventh asilomar conference on signals, systems & computers, 2003, vol 2. IEEE, pp 1398–1402

  37. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  38. Wang W, Wei C, Yang W, Liu J (2018) Gladnet: low-light enhancement network with global awareness. In: 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018). IEEE, pp 751–755

  39. Wang L, Fu G, Jiang Z, Ju G, Men A (2019) Low-light image enhancement with attention and multi-level feature fusion. In: 2019 IEEE international conference on multimedia & expo workshops (ICMEW). IEEE, pp 276–281

  40. Wang W, Xie E, Li X, Fan D-P, Song K, Liang D, Lu T, Luo P, Shao L (2021) Pyramid vision transformer: a versatile backbone for dense prediction without convolutions. arXiv:2102.12122

  41. Wei C, Wang W, Yang W, Liu J (2018) Deep retinex decomposition for low-light enhancement. arXiv:1808.04560

  42. Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  43. Xu K, Yang X, Yin B, Lau RWH (2020) Learning to restore low-light images via decomposition-and-enhancement. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2281–2290

  44. Yang C, Qiao S, Kortylewski A, Yuille A (2021) Locally enhanced self-attention: rethinking self-attention as local and context terms. arXiv:2107.05637

  45. Ying Z, Li G, Ren Y, Wang R, Wang W (2017) A new low-light image enhancement algorithm using camera response model. In: Proceedings of the IEEE international conference on computer vision workshops, pp 3015–3022

  46. Zamir SW, Arora A, Khan S, Hayat M, Khan FS, Yang M-H, Shao L (2020) Learning enriched features for real image restoration and enhancement. In: European conference on computer vision (ECCV) 2020: 16th, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXV 16. Springer, pp 492–51

  47. Zhang Y, Aydın TO (2021) Deep hdr estimation with generative detail reconstruction. In: Computer graphics forum, pp 179–190

  48. Zhang Q-L, Yang Y-B (2021) Sa-net: shuffle attention for deep convolutional neural networks. In: ICASSP 2021-2021 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 2235–2239

  49. Zhang Y, Zhang J, Guo X (2019) Kindling the darkness: a practical low-light image enhancer. In: Proceedings of the 27th ACM international conference on multimedia, pp 1632–1640

  50. Zhang C, Yan Q, Zhu Y u, Li X, Sun J, Zhang Y (2020) Attention-based network for low-light image enhancement. In: IEEE international conference on multimedia and expo (ICME). IEEE, pp 1–6

  51. Zhang Y, Di X, Zhang B, Li Q, Yan S, Wang C (2021) Self-supervised low light image enhancement and denoising. arXiv:2103.00832

  52. Zhao T, Wu X (2019) Pyramid feature attention network for saliency detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR)

  53. Zhuang L, Guan Y (2018) Adaptive image enhancement using entropy-based subhistogram equalization. Comput Intell Neurosci 2018

Download references

Funding

This work was supported in part by the National Natural Science Foundation of China under grant 62072169 and 62172156, and Natural Science Foundation, and Natural Key R&D Program of China under Grant No.2020YFB1713003,and Scientific Research Project of Hunan Provincial Education Department No.19A286.

Author information

Authors and Affiliations

  1. Hunan University, ChangSha, China

    Qiao Li, Bin Jiang & Chao Yang

  2. Beijing Institute of Radiation Medicine, Beijing, China

    Xiaochen Bo

  3. Shenzhen University, Shenzhen, China

    Xu Wu

Authors
  1. Qiao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bin Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaochen Bo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Jiang.

Ethics declarations

Conflict of Interests

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Q., Jiang, B., Bo, X. et al. Effective low-light image enhancement with multiscale and context learning network. Multimed Tools Appl 82, 15271–15286 (2023). https://doi.org/10.1007/s11042-022-13830-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13830-y

Keywords

Search

Navigation