Skip to main content
SpringerLink
Log in

Low-light image enhancement based on GAN with attention mechanism and color Constancy

  • 1203: Applications of Advanced Artificial Intelligence in Multimedia and Information Security
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Images captured in low-light often suffer from severe quality degraded problems, such as low contrast and color distortion, which make it intractable for further computer vision tasks. To solve the problems above, we proposed a trainable parallel network including Brightness Enhancement Module based on GAN and Color Fidelity Module, which are guided by attention mechanism and color constancy respectively. The experimental results show that the proposed method could effectively improve the image contrast and preserve the color. The proposed method performs better than the state-of-the-art image enhancement methods (e. g. GAN based method) for improving the quantitative assessment including PSNR (19.72, ↑2.6%), BIQI (71.37, ↑1%), CIEDE2000 (4.97, ↓52%) and Pearson Correlation Coefficient (0.86, ↑105%).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Ali A, Zhu Y, Chen Q et al (2019) Leveraging Spatio-temporal patterns for predicting citywide traffic crowd flows using deep hybrid neural networks[C]// 2019 IEEE 25th international conference on parallel and distributed systems (ICPADS). IEEE

  2. Ali A, Zhu Y, Zakarya M (2021) A data aggregation based approach to exploit dynamic spatio-temporal correlations for citywide crowd flows prediction in fog computing[J]. Multimed Tools Appl 2:31401–31433

    Article  Google Scholar 

  3. Babu KK, Dubey SR (2021) CSGAN: Cyclic-Synthesized Generative Adversarial Networks for image-to-image transformation[J]. Expert Syst Appl 169(169):114431. https://doi.org/10.1016/j.eswa.2020.114431

    Article  Google Scholar 

  4. Baiju PS, George SN (2021) l1/2 regularized joint low rank and sparse recovery technique for illumination map estimation in low light image enhancement[J]. Ambient Intell Human Comput 13:903–920. https://doi.org/10.1007/s12652-021-02947-x

  5. Brock A, Donahue J, Simonyan K (2018) Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096

  6. Cao HY, Liu CM, Shen XL et al Low illumination image processing based on adaptive threshold and local tone mapping[J/OL]. Laser & Optoelectronics Progress:1–12[2021-04-24].http://kns.cnki.net/kcms/detail/31.1690.TN.20200923.1151.002.html.

  7. Chen Q-j, Qu M (2021) Low-light image enhancement based on dual-residual convolutional network[J]. Chin J Liq Cryst Displays 36(2):305–316. https://doi.org/10.37188/CJLCD.2020-0168

    Article  Google Scholar 

  8. Chen C, Chen Q, Xu J et al. 2018 Learning to see in the dark[C]// 2018 IEEE/CVF conference on computer vision and pattern recognition. IEEE

  9. Cheng YM, Liu ZH (2020) X-ray image processing based on CLAHE fusion edge detail enhancement algorithm[J]. J Chengdu Univ Inf Technol 35(6):621–624

    Google Scholar 

  10. Endo H, Taguchi A (2019) A color image enhancement method by using the hue component[C]// 2019 19th international symposium on communications and information technologies (ISCIT). IEEE

  11. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Proces Syst:2672–2680

  12. Guo X, Li Y, Ling H (2017) LIME: low-light image enhancement via illumination map estimation[J]. IEEE Trans Image Process 26(2):982–993

    Article  MathSciNet  Google Scholar 

  13. He W (2015) Low-light image enhancement based on improve histogram [J]. Comput Sci 42(S1):241–242 262

    MathSciNet  Google Scholar 

  14. Huang HX, Chen FH (2020) Low-illumination image enhancement method based on attention mechanism and Retinex[J]. Laser Optoelectron Progress 57(20):53–60

    Google Scholar 

  15. Huang H, Tao HJ, Wang HF (2019) Low-illumination image enhancement using a conditional generative adversarial network. J Image Graph 24(12):2149–2158

    Google Scholar 

  16. Iqbal M, Ali SS, Riaz MM et al (2020) Color and white balancing in low-light image enhancement[J]. Optik 209:164260. https://doi.org/10.1016/j.ijleo.2020.164260

    Article  Google Scholar 

  17. Ji X-x, Zhang G (2017) Image fusion method of SAR and infrared image based on Curvelet transform with adaptive weighting[J]. Multimed Tools Appl 76(17):17633–17649

    Article  Google Scholar 

  18. Jia XY (2019) Denoising and enhancement algorithm of night low-light image based on wavelet transform[J]. Inf Technol Inf 2:107–109

    Google Scholar 

  19. Jiang Y, Gong X, Liu D, Cheng Y, Fang C, Shen X, Yang J, Zhou P, Wang Z (2019) Enlightengan: Deep light enhancement without paired supervision. arXiv preprint arXiv:1906.06972

  20. Jiang Y, Xu J, Yang B, Xu J, Zhu J (2020) Image Inpainting based on generative adversarial networks. IEEE Access 8:22884–22892. https://doi.org/10.1109/ACCESS.2020.2970169

    Article  Google Scholar 

  21. Jin Y, Zhang J, Li M, Tian Y, Zhu H, Fang Z (2017) Towards the Automatic Anime Characters Creation with Generative Adversarial Networks. arXiv preprint arXiv:1708.05509

  22. Jobson DR, Rahman Z, Woodell G (1997) A multi - scale retinex for bridging the gap between color images and the human observation of scenes[J]. IEEE Trans Image Process 6(7):965–976

    Article  Google Scholar 

  23. Kingma D, Ba J (2014) Adam: A method for stochastic optimization. [Online]. Available: https://arxiv.org/abs/1412.6980.

  24. Lee H, Sohn K, Min D (2020) Unsupervised low-light image enhancement using Bright Channel prior[J]. IEEE Signal Process Lett 27:251–255

    Article  Google Scholar 

  25. Li Q, Liu Q (2015) Adaptive enhancement algorithm for low illumination images based on wavelet transform[J]. Chin J Lasers 42(2):272–278. https://doi.org/10.3788/cjl201542.0209001

    Article  Google Scholar 

  26. Meng Y, Kong D, Zhu Z, Zhao Y (2019) From night to day: GANs based low quality image enhancement[J]. Neural Process Lett 50(1):799–814

    Article  Google Scholar 

  27. Pang XL, He ZH, Wang X et al (2020) Low-light inspection image enhancement based on histogram equalization algorithm[J]. Plant Maint Eng 18:76–77

    Google Scholar 

  28. Parihar AS, Verma OP, Khanna C (April 2017) Fuzzy-contextual contrast enhancement. IEEE Trans Image Process 26(4):1810–1819. https://doi.org/10.1109/TIP.2017.2665975

    Article  MathSciNet  Google Scholar 

  29. Ren W, Liu S, Ma L, Xu Q, Xu X, Cao X, du J, Yang MH (2019) Low-light image enhancement via a deep hybrid network[J]. IEEE Trans Image Process 28:4364–4375

    Article  MathSciNet  Google Scholar 

  30. Shi Z, Zhu MM, Guo B, Zhao M, Zhang C (2018) Nighttime low illumination image enhancement with single image using bright/dark channel prior[J]. Eurasip J Image Video Process 2018(1):13

    Article  Google Scholar 

  31. Shin YG, Sagong MC, Yeo YJ et al (2018) Adversarial Context Aggregation Network for Low-Light Image Enhancement[C]// 2018 Digital Image Computing: Techniques and Applications (DICTA)

  32. Singh K, Seth A, Sandhu HS, et al (2019) A comprehensive review of convolutional neural network based image enhancement techniques[C]// 2019 IEEE international conference on system, computation, automation and networking (ICSCAN). IEEE

  33. Tao L, Zhu C, Xiang G, et al (2018) LLCNN: a convolutional neural network for low-light image enhancement[C]// Visual Communications & Image Processing. IEEE

  34. Wang X, Chen L (2017) An effective histogram modification scheme for image contrast enhancement[J]. Signal Process Image Commun 58:187–198

    Article  Google Scholar 

  35. Wang T-C, Liu M-Y, Zhu J-Y, Tao A, Kautz J, Catanzaro B (2017) High-resolution image synthesis and semantic manipulation with conditional gans. arXiv: 1711.11585 [cs.CV]

  36. Wang Y, Huang Q, Hu J (2018) Adaptive Enhancement for non-uniform illumination Images via Pixel-wise Histogram Modification and Color Reconstruction. 220–224. https://doi.org/10.1109/SIPROCESS.2018.8600429.

  37. Wei C, Wang W, Yang W, Liu J (2018) Deep retinex decomposition for low-light enhancement. arXiv preprint arXiv:1808.04560

  38. Wei X, Liu D, Shen X, Fang C, Luo J (2020) Unsupervised Real-world Low-light Image Enhancement with Decoupled Networks. ArXiv abs/2005.02818.

  39. Xu W, Lee M, Zhang Y, You J, Suk S, Choi J (2018) Deep Residual Convolutional Network for Natural Image Denoising and Brightness Enhancement. 2018 International conference on platform technology and service (PlatCon), Jeju, pp. 1–6. https://doi.org/10.1109/PlatCon.2018.8472743

  40. Xu M, Li H, Chen J Extreme low-light image enhancement using deep neural network. Proc. SPIE 11720, Twelfth International Conference on Graphics and Image Processing (ICGIP 2020), 1172015 (27 January 2021). https://doi.org/10.1117/12.2589358

  41. Yan L, Fu J, Wang C, Ye Z, Chen H, Ling H (2021) Enhanced network optimized generative adversarial network for image enhancement. Multimed Tools Appl 80:14363–14381. https://doi.org/10.1007/s11042-020-10310-z

    Article  Google Scholar 

  42. Ying Z, Ge L, Ren Y, Wang R, Wang W (2018) A new low-light image enhancement algorithm using camera response model. In: IEEE international conference on computer vision workshop

  43. Zhang JX, Yang H (2021) ENHANCEMENT OF LOW ILLUMINATION COLOR IMAGE BASED ON HOMOMORPHIC HIGH-LOW PASS FILTERING AND MULTISCALE RETINEX[J]. Comput Appl Softw 38(01):232–237

    Google Scholar 

  44. Zhang N, Wang C, Zhao Y et al (2019) Deep tone mapping network in HSV color space[C]// 2019 IEEE visual communications and image processing (VCIP). IEEE

  45. Zhang H, Sindagi V, Patel VM (2020) Image De-raining using a conditional generative adversarial network. IEEE Trans Circuits Syst Video Technol 30(11):3943–3956. https://doi.org/10.1109/TCSVT.2019.2920407

    Article  Google Scholar 

  46. Zhong Z, Li J, Clausi DA, Wong A (July 2020) Generative adversarial networks and conditional random fields for hyperspectral image classification. IEEE Transact Cybern 50(7):3318–3329. https://doi.org/10.1109/TCYB.2019.2915094

    Article  Google Scholar 

  47. Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. CoRR abs/1703.10593. Available at http://arxiv.org/abs/1703.10593.

Download references

Acknowledgements

This work was supported by Key Lab of Intelligent and Green Flexographic Printing [grant number ZBKT202108].

Author information

Authors and Affiliations

  1. University of Shanghai for Science and Technology, Shanghai, 200093, China

    Xiaohong Wang, Yanxiu Zhai, Xiangcai Ma, Jing Zeng & Youci Liang

Authors
  1. Xiaohong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanxiu Zhai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiangcai Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jing Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Youci Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanxiu Zhai.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Zhai, Y., Ma, X. et al. Low-light image enhancement based on GAN with attention mechanism and color Constancy. Multimed Tools Appl 83, 3133–3151 (2024). https://doi.org/10.1007/s11042-022-13335-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13335-8

Keywords

Search

Navigation