Skip to main content
SpringerLink
Log in

Low-light image enhancement with joint illumination and noise data distribution transformation

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Images captured in low-light environments often face two problems: low contrast and high noise, which are caused by the low number of photons in the environment. Existing low-light image enhancement methods mainly focus on the previous problem to increase visibility while the latter one is usually addressed with a post-processing module. However, there is a coupling relationship between illumination and noise, and ignoring it will result in under-/over-smoothing of the enhanced images. To solve this problem, we propose a novel low-light image enhancement method based on simultaneous adjustment on illumination and noise using unpaired data. In other words, we consider illumination and noise as a joint data distribution. The proposed method consists of two main branches: a Distribution Extraction branch which is used to extract the joint distribution of illumination and noise in normal-light images, and a Distribution Transformation branch which transforms the low-light images in a spatial domain through the joint distribution. Extensive experiment results show that the proposed model can reach the network capability that trained with rich paired data and achieved satisfactory results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Li, M., Zhao, L., Zhou, D., Nie, R., Liu, Y., Wei, Y.: AEMS: an attention enhancement network of modules stacking for lowlight image enhancement. Vis. Comput. (2021)

  2. Hu, W., Wang, T., Wang, Y., Chen, Z., Huang, G.: Le–msfe–ddnet: a defect detection network based on low-light enhancement and multi-scale feature extraction. Vis. Comput. 1–15( 2021)

  3. Rahman, Z., Pu, Y.-F., Aamir, M., Wali, S.: Structure revealing of low-light images using wavelet transform based on fractional-order denoising and multiscale decomposition. Vis. Comput. 37(5), 865–880 (2021)

    Article  Google Scholar 

  4. Chen, C., Chen, Q., Xu, J., Koltun, V.: Learning to see in the dark. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3291–3300. IEEE (2018)

  5. Wang,W., Wei, C., Yang, W., Liu, J.: Gladnet: low-light enhancement network with global awareness. In Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition, pp. 751–755. IEEE,2(018)

  6. Wang, R., Zhang, Q., Fu, C.-W., Shen, X., Zheng, W.-S., Jia, J.: Underexposed photo enhancement using deep illumination estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6849–6857. IEEE (2019)

  7. Deng, G.: A generalized unsharp masking algorithm. IEEE Trans. Image Process. 20(5), 1249–1261 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  8. Yang, K.-F., Zhang, X.-S., Li, Y.-J.: A biological vision inspired framework for image enhancement in poor visibility conditions. IEEE Trans. Image Process. 29, 1493–1506 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  9. Yang, w., Wang, S., Fang, Y., Wang, Y., Liu, J.: From fidelity to perceptual quality: a semi-supervised approach for low-light image enhancement. In: Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3063–3072. IEEE (2020)

  10. Li, M., Liu, J., Yang, W., Sun, X., Guo, Z.: Structure-revealing low-light image enhancement via robust retinex model. IEEE Trans. Image Process. 27(6), 2828–2841 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  11. Zhu, A., Zhang, L., Shen, Y., Ma, Y., Zhao, S., Zhou, Y.: Zero-shot restoration of underexposed images via robust retinex decomposition. In: Proceedings of the 2020 IEEE International Conference on Multimedia and Expo, pp. 1–6. IEEE (2020)

  12. Ren, X., Yang, W., Cheng, W.-H., Liu, J.: Lr3m: robust low-light enhancement via low-rank regularized retinex model. IEEE Trans. Image Process. 29, 5862–5876 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  13. Lv, F., Li, Y., Lu, F.: Attention guided low-light image enhancement with a large scale low-light simulation dataset. arXiv preprint arXiv:1908.00682 (2019)

  14. Wang, Y., Cao, Y., Zha, Z.-J., Zhang, J., Xiong Z., Zhang, W., Wu, F>: Progressive retinex: mutually reinforced illumination-noise perception network for low-light image enhancement. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2015–2023. ACM (2019)

  15. Wei, K., Fu, Y., Yang, J., Huang, H.: A physics-based noise formation model for extreme low-light raw denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2758–2767. IEEE (2020)

  16. Xiong, W., Liu, D., Shen, X., Fang, C., Luo, J.: Unsupervised real-world low-light image enhancement with decoupled networks. arXiv preprint arXiv:2005.02818 (2020)

  17. Xu, K., Yang, X., Yin, B., Lau, R.W.H.: Learning to restore low-light images via decomposition-and-enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2281–2290. IEEE (2020)

  18. Pisano, E.D., Zong, S., Hemminger, B.M., DeLuca, M., Johnston, R.E., Muller, K., Braeuning, M.P., Pizer, S.M.: Contrast limited adaptive histogram equalization image processing to improve the detection of simulated spiculations in dense mammograms. J. Digit. Imaging 11(4), 193 (1998)

    Article  Google Scholar 

  19. Pizer, S.M., Johnston, R.E., Ericksen, J.P., Yankaskas, B.C., Muller, K.E.: Contrast-limited adaptive histogram equalization: speed and effectiveness. In: [1990] Proceedings of the First Conference on Visualization in Biomedical Computing, pp. 337–338. IEEE Computer Society (1990)

  20. Dong, X., Wang, G., Pang, Y., Li, W., Wen, J., Meng, W., Lu, Y.: Fast efficient algorithm for enhancement of low lighting video. In: Proceedings of the 2011 IEEE International Conference on Multimedia and Expo, pp. 1–6. IEEE (2011)

  21. Zhang, X., Shen, P., Luo, L., Zhang, L., Song, J.: Enhancement and noise reduction of very low light level images. In: Proceedings of the 21st International Conference on Pattern Recognition, pp. 2034–2037. IEEE (2012)

  22. Wang, S., Zheng, J., Hu, H.-M., Li, B.: Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans. Image Process. 22(9), 3538–3548 (2013)

    Article  Google Scholar 

  23. Guo, X., Li, Y., Ling, H.: Lime: low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 26(2), 982–993 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  24. Fu, X., Zeng, D., Huang, Y., Zhang, X.-P., Ding, X.: A weighted variational model for simultaneous reflectance and illumination estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2782–2790. IEEE (2016)

  25. Cai, B., Xu, X., Guo, K., Jia, K., Hu, B., Tao, D.: A joint intrinsic-extrinsic prior model for retinex. In: Proceedings of the IEEE international Conference on Computer Vision, pp. 4000–4009. IEEE (2017)

  26. Lore, K.G., Akintayo, A., Sarkar, S.: Llnet: a deep autoencoder approach to natural low-light image enhancement. Pattern Recognit. 61, 650–662 (2017)

    Article  Google Scholar 

  27. Wei, C., Wang, W., Yang, W., Liu, J.: Deep retinex decomposition for low-light enhancement. In: Proceedings of the British Machine Vision Conference, p. 155. British Machine Vision Association (2018)

  28. Wang, J., Tan, W., Niu, X., Yan, B.: Rdgan: Retinex decomposition based adversarial learning for low-light enhancement. In: Proceedings of the 2019 IEEE International Conference on Multimedia and Expo, pp. 1186–1191. IEEE (2019)

  29. Fan, M., Wang, W., Yang, W., Liu, J.: Integrating semantic segmentation and retinex model for low-light image enhancement. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 2317–2325. ACM (2020)

  30. Meng, Z., Xu, R., Man Ho, C.: Gia-net: global information aware network for low-light imaging. In: Proceedings of the European Conference on Computer Vision, pp. 327–342. IEEE (2020)

  31. Wang, L.-W., Liu, Z.-S., Siu, W.-C., Lun, D.P.K.: Lightening network for low-light image enhancement. IEEE Trans. Image Process. 29, 7984–7996 (2020)

    Article  MATH  Google Scholar 

  32. Kwon, D., Kim, G., Kwon, J.: Dale: Dark region-aware low-light image enhancement. arXiv preprint arXiv:2008.12493 (2020)

  33. Ren, W., Liu, S., Ma, L., Xu, Q., Xu, X., Cao, X., Du, J., Yang, M.-H.: Low-light image enhancement via a deep hybrid network. IEEE Trans. Image Process. 28(9), 4364–4375 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  34. Zhang, Y., Di, X., Zhang, B., Wang, C.: Self-supervised image enhancement network: training with low light images only. arXiv preprint arXiv:2002.11300 (2020)

  35. Jiang, Y., Gong, X., Liu, D., Cheng, Y., Fang, C., Shen, X., Yang, J., Zhou, P., Wang, Z.: Enlightengan: deep light enhancement without paired supervision. IEEE Trans. Image Process. 30, 2340–2349 (2021)

    Article  Google Scholar 

  36. Li, C., Guo, C., Chen, C.L.: Learning to enhance low-light image via zero-reference deep curve estimation. IEEE Trans. Pattern Anal. Mach. Intell. 1–1 (2021)

  37. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)

  38. Shen, Y., Gu, J., Tang, X., Zhou, B.: Interpreting the latent space of GANs for semantic face editing. In: Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9243–9252. IEEE (2020)

  39. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241. MICCAI (2015)

  40. Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. IEEE Trans. Pattern Anal. Mach. Intell. 1–1 (2020)

  41. Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the 2017 IEEE International Conference on Computer Vision, pp. 1510–1519. IEEE (2017)

  42. Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2261–2269. IEEE (2017)

  43. Zhu, J.-Y., Park, T., Isola, P., Efros, A. A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232. IEEE (2017)

  44. Ma, K., Zeng, K., Wang, Z.: Perceptual quality assessment for multi-exposure image fusion. IEEE Trans. Image Process. Publ. IEEE Signal Process. Soc. 24(11), 3345 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  45. Peng, Y., Chee, L., Chan, S.: Getting to know low-light images with the exclusively dark dataset. Comput. Vis. Image Underst. 178, 30–42 (2019)

    Article  Google Scholar 

  46. Cai, J., Gu, S., Zhang, L.: Learning a deep single image contrast enhancer from multi-exposure images. IEEE Trans. Image Process. 27(4), 2049–2062 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  47. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Proceedings of the European Conference on Computer Vision, pp. 694–711. Springer (2016)

  48. Ulyanov, D., Vedaldi, A., Lempitsky, V.: Improved texture networks: maximizing quality and diversity in feed-forward stylization and texture synthesis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6924–6932. IEEE (2017)

  49. Murray, N., Marchesotti, L., Perronnin, F.: Ava: a large-scale database for aesthetic visual analysis. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2408–2415. IEEE (2012)

  50. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034. IEEE (2015)

  51. Guo, C., Li, C., Guo, J., Loy, C.C., Hou, J., Kwong, S., Cong, R.: Zero-reference deep curve estimation for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1780–1789. IEEE (2020)

  52. Zhang, Y., Zhang, J., Guo, X.: Kindling the darkness: a practical low-light image enhancer. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 1632–1640. ACM (2019)

  53. Chan, L.C., Whiteman, P.: Hardware-constrained hybrid coding of video imagery. IEEE Trans. Aerosp. Electron. Syst. 1, 71–84 (1983)

    Article  Google Scholar 

  54. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  55. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local Nash equilibrium. In: Proceedings of the Advances in Neural Information Processing Systems, vol. 30, pp. 6626–6637. MIT Press (2017)

  56. Kurt, M.: Gensss: a genetic algorithm for measured subsurface scattering representation. Vis. Comput. 37(2), 307–323 (2021)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Natural Science Foundation of China (U1803262, 61602349, 61440016).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, 430065, China

    Sheng Guo, Wei Wang, Xiao Wang & Xin Xu

  2. Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, Wuhan, 430065, China

    Wei Wang, Xiao Wang & Xin Xu

Authors
  1. Sheng Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, S., Wang, W., Wang, X. et al. Low-light image enhancement with joint illumination and noise data distribution transformation. Vis Comput 39, 1363–1374 (2023). https://doi.org/10.1007/s00371-022-02412-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02412-6

Keywords

Search

Navigation