Skip to main content
SpringerLink
Log in

Subjective low-light image enhancement based on a foreground saliency map model

  • 1193: Intelligent Processing of Multimedia Signals
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Most existing low-light image enhancement methods enhance whole low-light image indiscriminately with the neglect of its subjective content, which may lead to over-enhancement and noise amplification problems in background. In this paper, we explore the challenging subjective low-light image enhancement problem. To this end, we first develop a novel foreground saliency detection model to measure the subjective content of low-light images. It is achieved by learning a saliency map and a depth map of low-light images based on CNN technique, and then fusing the two maps based on the Guided filter. Then, we incorporate the foreground saliency map model into a general retinex-based low-light image enhancement framework. Experimental results show that the proposed method well improves the subjective perception of low-light images without amplifying the noise in background compared with existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Abdullah-Al-Wadud M, Kabir MH, Dewan MA et al (2007) A dynamic histogram equalization for image contrast enhancement. IEEE Trans Consum Electron 53(2):593–600

    Article  Google Scholar 

  2. Ban SW, Jang YM, Lee M (2011) Affective saliency map considering psychological distance. Neurocomputing 74(11):1916–1925

    Article  Google Scholar 

  3. Borji A, Cheng MM, Jiang H et al (2015) Salient object detection: a benchmark. IEEE Trans Image Process 24(12):5706–5722

    Article  MathSciNet  Google Scholar 

  4. Bruce N, Tsotsos J (2007) Attention based on information maximization. J Vis 7(9):950–950

    Article  Google Scholar 

  5. Cai J, Gu S, Zhang L (2018) Learning a deep single image contrast enhancer from multi-exposure images. IEEE Trans Image Process 27(4):2049–2062

    Article  MathSciNet  Google Scholar 

  6. Fan Q, Qi C (2016) Saliency detection based on global and local short-term sparse representation. Neurocomputing 175:81–89

    Article  Google Scholar 

  7. Fan DP, Lin Z, Zhang Z et al (2020) Rethinking RGB-D salient object detection: models, data sets, and large-scale benchmarks. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2020.2996406

    Article  Google Scholar 

  8. Fu X, Zeng D, Huang Y, et al (2016) A weighted variational model for simultaneous reflectance and illumination estimation. In: Proceedings of 2016 IEEE conference on computer vision and pattern recognition (CVPR)

  9. Fu K, Zhao Q, Gu IYH, Yang J (2019) Deepside: a general deep framework for salient object detection. Neurocomputing 356:69–82

    Article  Google Scholar 

  10. Fu K, Fan DP, Ji GP, Zhao Q (2020) Jl-dcf: joint learning and densely-cooperative fusion framework for rgb-d salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3052–3062

  11. Godard C, Mac Aodha O, Firman M, Brostow GJ (2019) Digging into self-supervised monocular depth estimation. In Proceedings of the IEEE international conference on computer vision, pp 3828–3838.

  12. Guo X, Li Y, Ling H (2017) Lime: low-light image enhancement via illumination map estimation. IEEE Trans Image Process 26(2):982–993

    Article  MathSciNet  Google Scholar 

  13. He K, Sun J, Tang X (2010) Guided image filtering. In: Proceedings of European conference on computer vision, pp 1–14

  14. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of 2016 IEEE conference on computer vision and pattern recognition (CVPR)

  15. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  16. Jiang M, Huang S, Duan J, Zhao Q (2015) SALICON: saliency in context. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)

  17. Jing H, He X, Han Q et al (2014) Saliency detection based on integrated features. Neurocomputing 129:114–121

    Article  Google Scholar 

  18. Jobson DJ, Rahman Z (1997) Properties and performance of a center/surround retinex. IEEE Trans Image Process 6(3):451–462

    Article  Google Scholar 

  19. Karel Z (1994) Contrast limited adaptive histogram equalization. Graphics Gems.

  20. Land EH (1977) The retinex theory of color vision. Sci Am 237(6):108–128

    Article  Google Scholar 

  21. Loh YP, Chan CS (2019) Getting to know low-light images with the exclusively dark dataset. Comput Vis Image Underst 178:30–42

    Article  Google Scholar 

  22. Lore KG, Akintayo A, Sarkar S (2015) Llnet: a deep autoencoder approach to natural low-light image enhancement. Pattern Recogn 61:650–662

    Article  Google Scholar 

  23. Marcella C, Lorenzo B, Giuseppe S, Rita C (2018) Predicting human eye fixations via an lstm-based saliency attentive model. IEEE Trans Image Process. https://doi.org/10.1109/TIP.2018.2851672

    Article  MathSciNet  Google Scholar 

  24. Murray N, Vanrell M, Otazu X, Parraga CA (2011) Saliency estimation using a non-parametric low-level vision model. In: Proceedings of 2011 IEEE computer vision and pattern recognition (CVPR), pp 433–440

  25. Pizer SM, Amburn EP, Austin JD et al (1987) Adaptive histogram equalization and its variations. Comput Vis Graph Image Process 39(3):355–368

    Article  Google Scholar 

  26. Rahman ZU, Woodell GA (2002) Multi-scale retinex for color image enhancement. In: Proceedings of IEEE international conference on image processing

  27. Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vision 47(1–3):7–42

    Article  Google Scholar 

  28. Shen L, Yue Z, Feng F, et al (2017) Msr-net:low-light image enhancement using deep convolutional network. Arxiv

  29. Tavakoli HR, Laaksonen J (2016) Bottom-up fixation prediction using unsupervised hierarchical models. In Proceedings of Asian conference on computer vision, pp. 287–302

  30. Wang C, Ye Z (2005) Brightness preserving histogram equalization with maximum entropy: a variational perspective. IEEE Trans Consum Electron 51(4):1326–1334

    Article  Google Scholar 

  31. Wang N, Li Q, Abd El-Latif AA (2012) An accurate iris location method for low quality iris images. In: Proceedings of fourth international conference on digital image processing (ICDIP 2012)

  32. Wang S, Zheng J et al (2013) Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans Image Process 22(9):3538–3548

    Article  Google Scholar 

  33. Wang Y, Wang H, Yin C, Dai M (2015) Biologically inspired image enhancement based on retinex. Neurocomputing 177(177):373–384

    Google Scholar 

  34. Wei C, Wang W, Yang W, et al (2018) Deep retinex decomposition for low-light enhancement. In: proceedings of British machine vision conference (BMVC)

  35. Yan H, Zhang S, Zhang Y, Zhang L (2018) Monocular depth estimation with guidance of surface normal map. Neurocomputing 280:86–100

    Article  Google Scholar 

  36. Yu F, Koltun V (2016) Multi-scale context aggregation by dilated convolutions. In: International conference on learning representations.

  37. Zhang M, Ren W, Piao Y, et al (2020) Select, supplement and focus for RGB-D saliency detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3472–3481

  38. Zhang T, Abd El-Latif AA, Wang N, et al (2012) A new image segmentation method via fusing NCut eigenvectors maps. In: Proceedings of fourth international conference on digital image processing (ICDIP 2012).

  39. Zhao JX, Cao Y, Fan DP, et al (2019) Contrast prior and fluid pyramid integration for RGBD salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3927–3936

  40. Zhao JX, Liu JJ, Fan DP, et al (2019) EGNet: edge guidance network for salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 8779–8788

Download references

Acknowledgements

This work was supported by the National Key R&D Program of China (Grant No. 2021YFB2401904).

Author information

Authors and Affiliations

  1. Xi’an Jiaotong University, Xi’an, 710049, People’s Republic of China

    Pengcheng Hao, Meng Yang & Nanning Zheng

Authors
  1. Pengcheng Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nanning Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meng Yang.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hao, P., Yang, M. & Zheng, N. Subjective low-light image enhancement based on a foreground saliency map model. Multimed Tools Appl 81, 4961–4978 (2022). https://doi.org/10.1007/s11042-021-11590-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11590-9

Keywords

Search

Navigation