Skip to main content
SpringerLink
Log in

Single-image Deraining via a channel memory network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Single image rain removal requires a large number of channel features and image texture in the process of rain removal. In this work, we propose a Channel Memory Network (CMN) for single-image rain removal, which is a multi-stage rain removing network structure similar to recurrent neural network. One Channel Memory Block (CMB) is also employed by CMN to extract rain streaks texture feature efficiently. CMB is able to focus on features on the channel and optionally selects useful information. In addition, Channel Attention Block (CAB) is adopted in skip connection to enhance the features from previous modules. Gated Recurrent Unit (GRU) reduces the loss in the process of parameters sharing and selectively transfers the feature to the next stage. The network structure shares a lot of parameters so that more rain streaks information can be used efficiently in the process of feature transmission. A quantity of experiments show that the performance of the proposed method is better than that of the state-of-the-art methods on five synthetic datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Yang Y, Lu H (2019) Single image deraining using a recurrent multi-scale aggregation and enhancement network. In: 2019 IEEE International conference on multimedia and expo (ICME). IEEE, pp 1378–1383

  2. Li Y, Tan RT, Guo X, Lu J, Brown MS (2016) Rain streak removal using layer priors. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2736–2744

  3. Zhang H, Patel VM (2017) Convolutional sparse and low-rank coding-based rain streak removal. In: 2017 IEEE Winter conference on applications of computer vision (WACV). IEEE, pp 1259–1267

  4. Zhang H, Sindagi V, Patel VM (2019) Image de-raining using a conditional generative adversarial network. IEEE Transactions on Circuits and Systems for Video Technology

  5. Fu X, Huang J, Ding X, Liao Y, Paisley J (2017) Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans Image Process 26(6):2944–2956

    Article  MathSciNet  MATH  Google Scholar 

  6. Yang W, Tan RT, Feng J, Liu J, Guo Z, Yan S (2017) Deep joint rain detection and removal from a single image. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1357–1366

  7. Fu X, Huang J, Zeng D, Huang Y, Ding X, Paisley J (2017) Removing rain from single images via a deep detail network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3855–3863

  8. Luo Y, Xu Y, Ji H (2015) Removing rain from a single image via discriminative sparse coding. In: Proceedings of the IEEE international conference on computer vision, pp 3397–3405

  9. Chen YL, Hsu CT (2013) A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In: Proceedings of the IEEE international conference on computer vision, pp 1968–1975

  10. Kang LW, Lin CW, Fu YH (2011) Automatic single-image-based rain streaks removal via image decomposition. IEEE Transactions on Image Processing 21(4):1742–1755

    Article  MathSciNet  MATH  Google Scholar 

  11. Huang DA, Kang LW, Wang YCF, Lin CW (2013) Self-learning based image decomposition with applications to single image denoising. IEEE Transactions on Multimedia 16(1):83–93

    Article  Google Scholar 

  12. Chen DY, Chen CC, Kang LW (2014) Visual depth guided color image rain streaks removal using sparse coding. IEEE Transactions on Circuits and Systems for Video Technology 24(8):1430–1455

    Article  Google Scholar 

  13. Eigen D, Krishnan D, Fergus R (2013) Restoring an image taken through a window covered with dirt or rain. In: Proceedings of the IEEE international conference on computer vision, pp 633–640

  14. Fan Z, Wu H, Fu X, Huang Y, Ding X (2018) Residual-guide network for single image deraining. In: Proceedings of the 26th ACM international conference on Multimedia, pp 1751–1759

  15. Pan J, Liu S, Sun D, Zhang J, Liu Y, Ren J, Li Z, Tang J, Lu H, Tai YW, et al (2018) Learning dual convolutional neural networks for low-level vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3070–3079

  16. Zhang H, Patel VM (2018) Density-aware single image de-raining using a multi-stream dense network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 695–704

  17. Zamir SW, Arora A, Khan S, Hayat M, Khan FS, Yang MH, Shao L (2021) Multi-stage progressive image restoration. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 14821–14831

  18. Lu X, Wang W, Danelljan M, Zhou T, Shen J, Van Gool L (2020) Video object segmentation with episodic graph memory networks. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16. Springer, pp 661–679

  19. Zhou T, Li L, Bredell G, Li J, Konukoglu E (2021) Quality-aware memory network for interactive volumetric image segmentation. arXiv:210610686

  20. Wei W, Meng D, Zhao Q, Xu Z, Wu Y (2019) Semi-supervised transfer learning for image rain removal. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3877–3886

  21. Yasarla R, Patel VM (2019) Uncertainty guided multi-scale residual learning-using a cycle spinning cnn for single image de-raining. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8405–8414

  22. Yang W, Liu J, Yang S, Guo Z (2019) Scale-free single image deraining via visibility-enhanced recurrent wavelet learning. IEEE Trans Image Process 28(6):2948–2961

    Article  MathSciNet  MATH  Google Scholar 

  23. Yu W, Huang Z, Zhang W, Feng L, Xiao N (2019) Gradual network for single image de-raining. In: Proceedings of the 27th ACM international conference on multimedia, pp 1795–1804

  24. Liu X, Suganuma M, Sun Z, Okatani T (2019) Dual residual networks leveraging the potential of paired operations for image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7007–7016

  25. Du Y, Xu J, Zhen X, Cheng MM, Shao L (2020a) Conditional variational image deraining. IEEE Transactions on Image Processing

  26. Du Y, Xu J, Qiu Q, Zhen X, Zhang L (2020b) Variational image deraining. In: The IEEE Winter conference on applications of computer vision, pp 2406–2415

  27. Yasarla R, Patel VM (2020) Confidence measure guided single image de-raining. IEEE Trans Image Process 29:4544–4555

    Article  MATH  Google Scholar 

  28. Ren D, Zuo W, Hu Q, Zhu P, Meng D (2019) Progressive image deraining networks: a better and simpler baseline. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3937–3946

  29. Wei Y, Zhang Z, Zhang H, Hong R, Wang M (2019) A coarse-to-fine multi-stream hybrid deraining network for single image deraining. In: 2019 IEEE International Conference on Data Mining (ICDM). IEEE, pp 628–637

  30. Wang G, Sun C, Sowmya A (2019a) Erl-net: Entangled representation learning for single image de-raining. In: Proceedings of the IEEE international conference on computer vision, pp 5644– 5652

  31. Wang Z, Li J, Song G (2019b) Dtdn: Dual-task de-raining network. In: Proceedings of the 27th ACM international conference on multimedia, pp 1833–1841

  32. Zhu H, Wang C, Zhang Y, Su Z, Zhao G (2020) Physical model guided deep image deraining. In: 2020 IEEE International Conference on Multimedia and Expo (ICME). IEEE, pp 1–6

  33. Jiang K, Wang Z, Yi P, Chen C, Huang B, Luo Y, Ma J, Jiang J (2020) Multi-scale progressive fusion network for single image deraining. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8346–8355

  34. Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3146–3154

  35. Huang Z, Wang X, Huang L, Huang C, Wei Y, Liu W (2019) Ccnet: Criss-cross attention for semantic segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 603–612

  36. Li X, Wu J, Lin Z, Liu H, Zha H (2018) Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Proceedings of the European conference on computer vision (ECCV), pp 254–269

  37. Dai T, Cai J, Zhang Y, Xia ST, Zhang L (2019) Second-order attention network for single image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11065–11074

  38. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision (ECCV), pp 286–301

  39. Zhao H, Zhang Y, Liu S, Shi J, Loy CC, Lin D, Jia J (2018) Psanet: Point-wise spatial attention network for scene parsing. In: Proceedings of the European conference on computer vision (ECCV), pp 267–283

  40. Hore A, Ziou D (2010) Image quality metrics: Psnr vs. ssim. In: 2010 20th international conference on pattern recognition. IEEE, pp 2366–2369

  41. Charbonnier P, Blanc-Feraud L, Aubert G, Barlaud M (1994) Two deterministic half-quadratic regularization algorithms for computed imaging. In: Proceedings of 1st international conference on image processing, vol 2. IEEE, pp 168–172

Download references

Acknowledgements

We are very grateful for the great support from the development of the deep learning model for industrial intelligent quality inspection(CQ20210053).

Author information

Authors and Affiliations

  1. Department of Vision Algorithm, Changzhou Micro-Intelligence Co. Ltd., Changzhou, China

    Yan Zhang, Jun Guo & Jianqing Li

  2. Shanghai University of Engineering Science, Shanghai, 201620, China

    Juan Zhang

Authors
  1. Yan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianqing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Juan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Guo.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Guo, J., Li, J. et al. Single-image Deraining via a channel memory network. Appl Intell 53, 1009–1020 (2023). https://doi.org/10.1007/s10489-022-03441-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03441-3

Keywords

Search

Navigation