Skip to main content
SpringerLink
Log in

A joint deep neural networks-based method for single nighttime rainy image enhancement

  • Deep Learning & Neural Computing for Intelligent Sensing and Control
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In rainy conditions, especially at night with low illumination, the visual of images obtained by outdoor computer vision systems is degraded significantly, leading to a significant negative effect on the work of the outdoor computer vision system. In this paper, we develop a new rainy image model to describe rain scenes at night with low illumination. From this model, we propose a joint deep neural network-based method for single nighttime rainy image enhancement. First, a decom-net based on Retinex theory is employed for image decomposition, and the purpose of this sub-net is to extract the reflection image and the illumination image from the input image. Then, an enhancement net is proposed for illumination adjustment. The goal of this sub-net is to remove the negative effect (low visual) caused by low illumination. Finally, a symmetric sub-net termed multi-stream network-based contextual autoencoder is developed, where rain features are directly learned from the enhanced nighttime rainy images in a recurrent way. The goal of this sub-net is to effectively remove rain streaks from the illumination-enhanced image. The experimental results show the advantage and effectiveness of the proposed method, and evident improvements over existing state-of-the-art methods are obtained with the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Qian R, Tan RT, Yang W, Su J, Liu J (2018) Attentive generative adversarial network for raindrop removal from a single image. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2482–2491

  2. Zhang H, Patel VM (2018) Density-aware single image de-raining using a multi-stream dense network. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, pp 695–704

  3. You S, Tan RT, Kawakami R, Mukaigawa Y, Ikeuchi K (2016) Adherent raindrop modeling, detectionand removal in video. IEEE Trans Pattern Anal Mach Intell 38(9):1721–1733

    Article  Google Scholar 

  4. Kim JH, Lee C, Sim JY et al (2013) Single image deraining using an adaptive nonlocal means filter. In: Proceedings of 2013 20th IEEE international conference on image processing, Melbourne, pp 914–917

  5. Shi Z, Li Y, Zhao M, Feng Y, He L (2018) Weighted median guided filter for single image rain removal. EURASIP J Image Video Process 2018:35. https://doi.org/10.1186/s13640-018-0275-9

    Article  Google Scholar 

  6. Kang LW, Lin CW, Fu YH (2012) Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans Image Process 21(4):1742–1755

    Article  MathSciNet  Google Scholar 

  7. Luo Y, Xu Y, Ji H (2015) Removing rain from a single image via discriminative sparse coding. In: Proceedings of the IEEE international conference on computer vision, Santiago, pp 3397–3405

  8. Li Y, Tan RT, Guo X, Lu J, Brown MS (2017) Single image rain streak separation using layer priors. IEEE Trans Image Process 26(8):3874–3885

    Article  MathSciNet  Google Scholar 

  9. Shi Z, Li Y, Zhao M, Feng Y, He L (2018) Multi-stage filter for single image rain removal. IET Image Proc 12(10):1866–1872

    Article  Google Scholar 

  10. Fu X, Huang J, Ding X, Liao Y, Paisley J (2017) Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans Image Process 26(6):2944–2956

    Article  MathSciNet  Google Scholar 

  11. Yang W, Tan RT, Feng J, Liu J, Guo Z, Yan S (2017) Deep joint rain detection and removal from a single image. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1357–1366

  12. Wei C, Wang W, Yang W, Liu J (2018) Deep Retinex decomposition for low-light enhancement, British machine vision conference 2018, BMVC 2018, Northumbria University, Newcastle. https://arxiv.org/abs/1808.04560

  13. Li Y, Tan RT, Brown MS (2015) Nighttime haze removal with glow and multiple light colors. In: IEEE international conference on computer vision, pp 226–234

  14. Pei SC, Lee TY (2012) Nighttime haze removal using color transfer pre-processing and dark channel prior. In: IEEE international conference on image processing, pp 957–960

  15. Reinhard E, Ashikhmin M, Gooch B, Shirley P (2001) Color transfer between images. IEEE Comput Gr Appl 21(5):34–41

    Article  Google Scholar 

  16. Lin SCF, Wong CY, Rahman MA, Jiang G, Liu S, Kwok N et al (2015) Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng 46:356–370

    Article  Google Scholar 

  17. Zhou Z, Sang N, Hu X (2014) Global brightness and local contrast adaptive enhancement for low illumination color image. Optik Int J Light Electron Opt 125(6):1795–1799

    Article  Google Scholar 

  18. He K, Sun J, Tang X (2011) Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell 33(12):2341–2353

    Article  Google Scholar 

  19. Shi Z, Zhu M, Guo B, Zhao M (2017) A photographic negative imaging inspired method for low illumination night-time image enhancement. Multimed Tools Appl 76:15027–15048

    Article  Google Scholar 

  20. Wang Y, Zhuo S, Tao D, Bu J, Li N (2013) Automatic local exposure correction using bright channel prior for under-exposed images. Signal Process 93:3227–3238

    Article  Google Scholar 

  21. Guo X, Li Y, Ling H (2017) LIME: low-light image enhancement via illumination map estimation. IEEE Trans Image Process 26:982–993

    Article  MathSciNet  Google Scholar 

  22. Shi Z, Zhu M, Guo B, Zhao M (2018) Nighttime low illumination image enhancement with single image using bright/dark channel prior. EURASIP J Image Video Process 2018:13. https://doi.org/10.1186/s13640-018-0251-4

    Article  Google Scholar 

  23. Land EH (1986) Recent advances in retinex theory. Vis Res 26(1):7–21

    Article  Google Scholar 

  24. Land EH (1986) An alternative technique for the computation of the designator in the retinex theory of color vision. Proc Natl Acad Sci 83(10):3078–3080

    Article  Google Scholar 

  25. Jobson DJ, Rahman Z, Woodell GA (1997) Properties and performance of a center/surround retinex. IEEE Trans Image Process Publ IEEE Signal Process Soc 6(3):451–462

    Article  Google Scholar 

  26. Jobson DJ, Rahman ZU, Woodell GA (1997) A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans Image Process 6(7):965–976

    Article  Google Scholar 

  27. Guo X, Li Y, Ling H (2017) Lime: low-light image enhancement via illumination map estimation. IEEE Trans Image Process 26(2):982–993

    Article  MathSciNet  Google Scholar 

  28. Li M, Liu J, Yang W, Guo Z (2018) Structure-revealing low light image enhancement via robust retinex model. IEEE Trans Image Process 27(6):2828–2841

    Article  MathSciNet  Google Scholar 

  29. Wang S, Zheng J, Hu HM, Li B (2013) Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans Image Process 22(9):3538–3548

    Article  Google Scholar 

  30. Narasimhan SG, Nayar SK (2003) Contrast restoration of weather degraded images. IEEE Trans Pattern Anal Mach Intell 25(6):713–724

    Article  Google Scholar 

  31. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proc IEEE Int Conf Comput Vis 2:416–423

    Google Scholar 

  32. Garg K, Nayar SK (2006) Photorealistic rendering of rain streaks. ACM Trans Gr 25:996–1002

    Article  Google Scholar 

  33. Huynh-Thu Q, Ghanbari M (2008) Scope of validity of PSNR in image/video quality assessment. Electron Lett 44(13):800–801

    Article  Google Scholar 

  34. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  35. Khajwaniya KK, Tiwari V (2015) Satellite image denoising using Weiner filter with SPEA2 algorithm. In: IEEE 9th international conference on intelligent systems and control (ISCO), Coimbatore, pp 1–6

  36. Manu BN (2015) Rain removal from still images using L0 gradient minimization technique. In: 7th International conference on information technology and electrical engineering (ICITEE 2015), At Chiang Mai, Thailand, pp 263–268

Download references

Acknowledgement

This work was supported by Grant No. 61872290 from the National Natural Science Foundation of China.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Xi’an University of Technology, No. 5 South Jinhua Road, Xi’an, 710048, China

    Zhenghao Shi, Yaning Feng & Minghua Zhao

  2. School of Information Science and Technology, Aichi Prefectural University, 1522-3 Ibaragabasama, Nagakute, Aichi, 4801198, Japan

    Lifeng He

Authors
  1. Zhenghao Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yaning Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minghua Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lifeng He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenghao Shi.

Ethics declarations

Conflict of interest

We declare that no financial and personal relationships or organizations inappropriately affect our work and that no professional or other personal interests of any nature in any product, service, and/or company affect the position presented in, or the review of, this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, Z., Feng, Y., Zhao, M. et al. A joint deep neural networks-based method for single nighttime rainy image enhancement. Neural Comput & Applic 32, 1913–1926 (2020). https://doi.org/10.1007/s00521-019-04501-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04501-5

Keywords

Search

Navigation