Skip to main content
SpringerLink
Log in

Single Image Deraining by Fully Exploiting Contextual Information

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Single-image deraining is challenging due to the lack of temporal information. Current methods based on deep neural networks have achieved good performance in this task. However, these methods are still less effective in handling complex rainy situations. In this paper, we propose Channel-attention-based Multi-scale Recurrent Residual Network (CMRRNET), which tries to fully exploit contextual information of rainy images from multiple aspects. First, we construct a hybrid feature extraction module, which consists of the dilated convolution block and the multi-scale convolution block, to fully obtain image feature information. Second, we adopt the residual channel attention mechanism which makes the network aware of the importance of different channels. Third, we introduce long short-term memory to extract the correlation information of the features between different stages. We conduct extensive experiments on both synthetic and real rainy images. Ablation studies and extensive comparisons with state-of-the-art methods demonstrate the effectiveness of our CMRRNET.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Zhang X, Li H, Qi Y, Leow WK, Ng TK (2006) Rain removal in video by combining temporal and chromatic properties. In: 2006 IEEE international conference on multimedia and expo, pp 461–464

  2. Kim J-H, Sim J-Y, Kim C-S (2015) Video deraining and desnowing using temporal correlation and low-rank matrix completion. IEEE Trans Image Process 24(9):2658–2670

    Article  MathSciNet  Google Scholar 

  3. Liu J, Yang W, Yang S, Guo Z (2018) Erase or fill? Deep joint recurrent rain removal and reconstruction in videos. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3233–3242

  4. Chen J, Tan C-H, Hou J, Chau L-P, Li H (2018) Robust video content alignment and compensation for rain removal in a CNN framework. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6286–6295

  5. Kang L-W, Lin C-W, Fu Y-H (2012) Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans Image Process 21(4):1742–1755

    Article  MathSciNet  Google Scholar 

  6. Luo Y, Xu Y, Ji H (2015) Removing rain from a single image via discriminative sparse coding. In: Proceedings of the IEEE international conference on computer vision, pp 3397–3405

  7. Chen Y-L, Hsu C-T (2013) A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In: Proceedings of the IEEE international conference on computer vision, pp 1968–1975

  8. Zhang H, Patel VM (2017) Convolutional sparse and low-rank coding-based rain streak removal. In: 2017 IEEE winter conference on applications of computer vision, pp 1259–1267

  9. Li Y, Tan RT, Guo X, Lu J, Brown MS (2016) Rain streak removal using layer priors. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2736–2744

  10. Kim J-H, Lee C, Sim J-Y, Kim C-S (2013) Single-image deraining using an adaptive nonlocal means filter. In: 2013 IEEE international conference on image processing, pp 914–917

  11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  12. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2261–2269

  13. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  14. Yu F, Koltun V (2016) Multi-scale context aggregation by dilated convolutions. In: International conference on learning representations

  15. Yang W, Tan RT, Feng J, Liu J, Guo Z, Yan S (2017) Deep joint rain detection and removal from a single image. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1685–1694

  16. Li X, Wu J, Lin Z, Liu H, Zha H (2018) Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Proceedings of the European conference on computer vision, pp 262–277

  17. Ren D, Zuo W, Hu Q, Zhu P, Meng D (2019) Progressive image deraining networks: a better and simpler baseline. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3932–3941

  18. Yang W, Liu J, Yang S, Guo Z (2019) Scale-free single image deraining via visibility-enhanced recurrent wavelet learning. IEEE Trans Image Process 28(6):2948–2961

    Article  MathSciNet  Google Scholar 

  19. Fu X, Huang J, Zeng D, Huang Y, Ding X, Paisley J (2017) Removing rain from single images via a deep detail network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1715–1723

  20. Zhang H, Patel VM (2018) Density-aware single image de-raining using a multi-stream dense network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 695–704

  21. Li G, He X, Zhang W, Chang H, Dong L, Lin L (2018) Non-locally enhanced encoder-decoder network for single image de-raining. In: Proceedings of the 26th ACM international conference on multimedia, pp 1056–1064

  22. Jiang K, Wang Z, Yi P, Chen C, Huang B, Luo Y, Ma J, Jiang J (2020) Multi-scale progressive fusion network for single image deraining. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8346–8355

  23. Zhu H, Wang C, Zhang Y, Su Z, Zhao G (2020) Physical model guided deep image deraining. In: 2020 IEEE international conference on multimedia and expo, pp 1–6

  24. Wang C, Xing X, Wu Y, Su Z, Chen J (2020) DCSFN: deep cross-scale fusion network for single image rain removal. In: Proceedings of the 28th ACM international conference on multimedia, pp 1643–1651

  25. Yasarla R, Patel VM (2020) Confidence measure guided single image de-raining. IEEE Trans Image Process 29:4544–4555

    Article  Google Scholar 

  26. Deng S, Wei M, Wang J, Feng Y, Liang L, Xie H, Wang FL, Wang M (2020) Detail-recovery image deraining via context aggregation networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 14560–14569

  27. Qian R, Tan RT, Yang W, Su J, Liu J (2018) Attentive generative adversarial network for raindrop removal from a single image. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2482–2491

  28. Zhang H, Sindagi V, Patel VM (2020) Image de-raining using a conditional generative adversarial network. IEEE transactions on circuits and systems for video technology, pp 3943–3956

  29. Zhao Y, Zhai D, Jiang J, Liu X (2020) ADRN: attention-based deep residual network for hyperspectral image denoising. In: ICASSP 2020–2020 IEEE international conference on acoustics, speech and signal processing, pp 2668–2672

  30. Xingjian S, Chen Z, Wang H, Yeung D-Y, Wong W-K, Woo W-c (2015) Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Advances in neural information processing systems, pp 802–810

  31. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

  32. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision, pp 286–301

  33. Huynh-Thu Q, Ghanbari M (2008) Scope of validity of PSNR in image/video quality assessment. Electron Lett 44(13):800–801

    Article  Google Scholar 

  34. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  35. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in pytorch. In: NIPS autodiff workshop: the future of gradient-based machine learning software and techniques

  36. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: International conference on learning representations

  37. Wei C, Wang W, Yang W, Liu J (2018) Deep retinex decomposition for low-light enhancement. In: British machine vision conference

  38. Guo X, Li Y, Ling H (2016) LIME: low-light image enhancement via illumination map estimation. IEEE Trans Image Process 26(2):982–993

    Article  MathSciNet  Google Scholar 

  39. Zhang Y, Zhang J, Guo X (2019) Kindling the darkness: a practical low-light image enhancer. In: Proceedings of the 27th ACM international conference on multimedia, pp 1632–1640

  40. Guo Y, Wu Z, Shen D (2020) Learning longitudinal classification-regression model for infant hippocampus segmentation. Neurocomputing 391:191–198

    Article  Google Scholar 

  41. Chen L, Wu Z, Hu D, Wang Y, Mo Z, Wang L, Lin W, Shen D, Li G, Consortium UUBCP (2020) A deep spatial context guided framework for infant brain subcortical segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 646–656

  42. Hao S, Zhou Y, Guo Y (2020) A brief survey on semantic segmentation with deep learning. Neurocomputing 406:302–321

    Article  Google Scholar 

  43. Liu L, Ouyang W, Wang X, Fieguth P, Chen J, Liu X, Pietikäinen M (2020) Deep learning for generic object detection: a survey. Int J Comput Vis 128(2):261–318

    Article  Google Scholar 

Download references

Acknowledgements

This work is partially supported by the National Key Research and Development Program of China under Grant 2019YFA0706200, the National Nature Science Foundation of China under Grants 61772171, 61632007, 61876056, the Fundamental Research Funds for the Central Universities under Grants PA2020GDKC0023, PA2019GDZC0095, and the Shanghai Philosophy and Social Science Planning Project under Grant A1713.

Author information

Authors and Affiliations

  1. Hefei University of Technology, Hefei, China

    Xiaoxian Cao & Shijie Hao

  2. Shanghai Polytechnic University, Shanghai, China

    Lei Xu

Authors
  1. Xiaoxian Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shijie Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lei Xu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, X., Hao, S. & Xu, L. Single Image Deraining by Fully Exploiting Contextual Information. Neural Process Lett 54, 2613–2627 (2022). https://doi.org/10.1007/s11063-021-10486-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10486-x

Keywords

Search

Navigation