Skip to main content
SpringerLink
Log in

From Night to Day: GANs Based Low Quality Image Enhancement

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Nighttime images have lower contrast and higher noise than their daytime counterparts of the same scene. The aim of nighttime image enhancement is to improve the visual quality of nighttime images, so that they are visually as close as possible to daytime images. This problem is still challenging because of the deteriorated conditions of illumination lack and uneven lighting. In this paper, we propose a generative adversarial networks (GANs) based framework for nighttime image enhancement. To take advantage of GANs’ powerful ability of generating image from real data distribution, we make the established network well constrained by combining several loss functions including adversarial loss, perceptual loss, and total variation loss. Particularly, a pre-trained network is applied to leverage the perceptual loss which is beneficial to generate high-quality images. Meanwhile, for tackling the light-at-night effect, we present a fusion network in which the dark channel prior based illumination compensation is employed for the training of generator network. Experimental results have demonstrated the effectiveness of the proposed nighttime image enhancement network.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Ketcham DJ (1976) Real-time image enhancement techniques. OSA Image Process 74(2):120–125

    Article  Google Scholar 

  2. Rahman Z, Jobson DJ, Woodell GA (2002) Multi-scale retinex for color image enhancement. In: International conference on image processing, pp 1003–1006

  3. Jobson DJ, Rahman Z, Woodell GA (1997) A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans Image Process 6(7):965–976

    Article  Google Scholar 

  4. He K, Sun J, Tang X (2009) Single image haze removal using dark channel prior. In: IEEE conference on computer vision and pattern recognition, pp 1956–1963

  5. Dong X, Wang G, Pang Y, Li W, Wen J, Meng W, Lu Y (2011) Fast efficient algorithm for enhancement of low lighting video. In: IEEE international conference on multimedia and expo, pp 1–6

  6. Jiang X, Yao H, Zhang S, Lu X, Zeng W (2013) Night video enhancement using improved dark channel prior. In: IEEE international conference on image processing, pp 553–557

  7. Gong Y, Lee Y, Nguyen TQ (2016) Nighttime image enhancement applying dark channel prior to raw data from camera. In: International SoC design conference, pp 173–174

  8. Iizuka S, Simo-Serra E, Ishikawa H (2017) Globally and locally consistent image completion. ACM Trans Graph 36(4):107:1–107:14

    Article  Google Scholar 

  9. Yeh RA, Chen C, Lim T, Hasegawa-Johnson M, Do MN (2017) Semantic image inpainting with perceptual and contextual losses. In: IEEE conference on computer vision and pattern recognition

  10. Taigman Y, Polyak A, Wolf L (2017) Unsupervised cross-domain image generation. In: International conference on learning representations

  11. Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken AP, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: IEEE conference on computer vision and pattern recognition, pp 105–114

  12. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition, pp 770–778

  13. Zhu J, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE international conference on computer vision, pp 2242–2251

  14. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Conference on neural information processing systems, pp 2672–2680

  15. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: International conference on machine learning, pp 214–223

  16. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of Wasserstein GANs. In: Conference on neural information processing systems, pp 5769–5779

  17. Liu M, Breuel T, Kautz J (2017) Unsupervised image-to-image translation networks. In: Conference on neural information processing systems, pp 700–708

  18. Huang R, Zhang S, Li T, He R (2017) Beyond face rotation: global and local perception GAN for photorealistic and identity preserving frontal view synthesis. In: IEEE international conference on computer vision, pp 2458–2467

  19. Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: International conference on machine learning, vol 30, p 1

  20. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pp 448–456

  21. Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. In: International conference on learning representations

  22. Johnson J, Alahi A, Li F (2016) Perceptual losses for real-time style transfer and super-resolution. In: European conference on computer vision, pp 694–711

  23. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations

  24. Deng J, Dong W, Socher R, Li L, Li K, Li F (2009) ImageNet: a large-scale hierarchical image database. In: IEEE conference on computer vision and pattern recognition, pp 248–255

  25. Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Phys D Nonlinear Phenom 60:259–268

    Article  MathSciNet  MATH  Google Scholar 

  26. Strong D, Chan T (2003) Edge-preserving and scale-dependent properties of total variation regularization. Inverse Probl 19(6):165–187

    Article  MathSciNet  MATH  Google Scholar 

  27. Milford M, Wyeth GF (2012) SeqSLAM: visual route-based navigation for sunny summer days and stormy winter nights. In: IEEE international conference on robotics and automation, pp 1643–1649

  28. Bermejo S, Cabestany J (2001) Oriented principal component analysis for large margin classifiers. Neural Netw 14(10):1447–1461

    Article  MATH  Google Scholar 

  29. Isola P, Zhu J, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: IEEE conference on computer vision and pattern recognition, pp 5967–5976

Download references

Acknowledgements

The authors would like to thank the reviewers for their valuable suggestions. This work was in part supported by the National Natural Science Foundation of China (Nos. 61572068, 61532005) and in part by the Fundamental Research Funds for the Central Universities under Grant No. 2018JBZ001.

Author information

Authors and Affiliations

  1. Institute of Information Science, Beijing Jiaotong University, Beijing, 100044, China

    Yingying Meng, Deqiang Kong, Zhenfeng Zhu & Yao Zhao

  2. Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing, 100044, China

    Yingying Meng, Deqiang Kong, Zhenfeng Zhu & Yao Zhao

  3. Microsoft, Beijing, 100080, China

    Deqiang Kong

Authors
  1. Yingying Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Deqiang Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhenfeng Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yao Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenfeng Zhu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meng, Y., Kong, D., Zhu, Z. et al. From Night to Day: GANs Based Low Quality Image Enhancement. Neural Process Lett 50, 799–814 (2019). https://doi.org/10.1007/s11063-018-09968-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-018-09968-2

Keywords

Search

Navigation