Skip to main content
SpringerLink
Log in

New Single Image Rain Removal Algorithm Based on Dual Parallel Branch Residual Overlay Network

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

The end-to-end convolutional neural network models have been widely used in single image de-raining, which can extract clean background images from rainy images. However, they suffer from gradient vanishing with increased network depth. With the aim of tackling this problem, this paper proposes an effective rain removal algorithm based on dual parallel branch residual overlay network (DBRONet). Firstly, the two parallel branches with different functions are combined with increasing the width of the network, which can reduce the depth of the network effectively. Secondly, the upper branch uses multi-scale rain streak extraction blocks (MRSEB) composed of multi-scale residual blocks and extracts rain streaks of different densities, sizes and directions in the rainy images. The lower branch utilizes dilated convolution attention residual block (DARB) to expand the receptive field and obtains more context information without increasing the depth of the network. Finally, the de-raining image is obtained by superposition features of the two branches and the original image. Experimental results on synthetic and real datasets show that DBRONet can effectively reduce the depth of the network and the number of parameters. Compared with the existing methods in terms of quantitative and qualitative indicators, it has achieved the most advanced results. When comparing with other methods on Rain100H, Rain100L, Rain12, Rain1400 with the improvements of 0.76 dB, 0.29 dB, 0.11 dB and 0.25 dB on PSNR value and 0.4%, 0.3%, 0.4%, 1.2% on SSIM value, respectively. The source code can be found at https://github.com/RemeberMeX/DBRONet.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. J. Chen, L.P. Chau, A rain pixel recovery algorithm for videos with highly dynamic scenes. IEEE Trans. Image Process. 23(3), 1097–1104 (2014)

    Article  MathSciNet  Google Scholar 

  2. Y.L. Chen, C.T. Hsu, in Proceedings of IEEE International Conference on Computer Vision (ICCV). A generalized low-rank appearance model for spatio-temporally correlated rain streaks (2013), pp. 1968–1975

  3. Y. Chang, L.X. Yan, S. Zhong, in Proceedings of IEEE International Conference on Computer Vision (ICCV). Transformed low-rank model for line pattern noise removal (2017), pp. 1726–1734

  4. D.D Chen, M.M. He, Q.N Fan, J. Liao, L.H. Zhang, D.D. Hou, L.Yuan,G. Hua, in Proceedings of IEEE Winter Conference on Applications of Computer Vision (WACV). Gated context aggregation network for image dehazing and deraining (2019), pp.1375–1383

  5. L. Cai, S. Li, D. Ren, P. Wang, in Proceedings of IEEE International Conference on Image Processing (ICIP). Dual Recursive Network for Fast Image Deraining (2019), pp.2756–2760

  6. C. Dong, C.C. Loy, K.M. He, X.O. Tang, Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  7. X.Y. Fu, B. Ling, Y. Huang, X.H. Ding, J. Paisley, Lightweight pyramid networks for image deraining. IEEE Trans. Neural Netw. Learn. Syst. 31(6), 1794–1807 (2020)

    Article  Google Scholar 

  8. X.Y. Fu, J.B. Huang, X.H. Ding, Y.H. Liao, J. Paisley, Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26(6), 2944–2956 (2017)

    Article  MathSciNet  Google Scholar 

  9. S. Gai, Z. Bao, New image denoising algorithm via improved deep convolutional neural network with perceptive loss. Expert Syst. Appl. 138(8), 1–9 (2019)

    Google Scholar 

  10. S. Gai, B. Zhang, C.H. Yang, L. Yu, Speckle noise reduction in medical ultrasound image using monogenic wavelet and laplace mixture distribution. Dig. Sig. Process. 72(6), 192–207 (2018)

    Article  Google Scholar 

  11. S. Gai, G.W. Yang, S. Zhang, Multiscale texture classification using reduced quaternion wavelet transform. Int. J. Electron. Commun. 67(3), 233–241 (2013)

    Article  Google Scholar 

  12. S. Gai, G. Yang, M. Wan, Employing quaternion wavelet transform for banknote classification. Neurocomputing 118(8), 171–178 (2013)

    Article  Google Scholar 

  13. K.M. He, X.Y. Zhang, S.Q. Ren, J. Sun, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Deep residual learning for image recognition (2016), pp. 770–778

  14. J. Hu, L. Shen, G. Sun, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Squeeze and excitation networks (2018), pp. 7132–7141

  15. Q. Huynh-Thu, M. Ghanbari, Scope of validity of PSNR in image/video quality assessment. Electron. Lett. 44(13), 800–801 (2008)

    Article  Google Scholar 

  16. T.X. Jiang, T.Z. Huang, X.L. Zhao, L.J. Deng, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). A novel tensor-based video rain streaks removal approach via utilizing discriminatively intrinsic priors (2017), pp. 4057–4066

  17. J. Johnson, A. Alahi, F.F. Li, in Proceedings of Conference on European Conference on Computer Vision (ECCV). Perceptual losses for real-time style transfer and super-resolution (2016), pp. 694–711.

  18. J.H. Kim, B.G. Kim, P.P. Roy, D.M. Jeong, Efficient facial expression recognition algorithm based on hierarchical deep neural network structure. IEEE Access 7, 41273–41285 (2019)

    Article  Google Scholar 

  19. J.H. Kim, G.S. Hong, B.G. Kim, D.P. Dogra, Deepgesture: deep learning-based gesture recognition scheme using motion sensors. Disp. 55, 38–45 (2018)

    Article  Google Scholar 

  20. L.W. Kang, C.W. Lin, Y.H. Fu, Automatic single image based rain streaks removal via image decomposition. IEEE Trans. Image Process. 21(4), 1742–1755 (2012)

    Article  MathSciNet  Google Scholar 

  21. D. Kingma, J. Ba, in international conference for learning representations (ICLR). Adam: a method for stochastic optimization (2015)

  22. X. Li, J.L. Wu, Z.C Lin, H. Liu, H.B Zha, in Proceedings of Conference on European Conference on Computer Vision (ECCV). Recurrent squeeze and excitation context aggregation net for single image deraining(2018), pp. 254–269

  23. Y. Luo,Y. Xu, H. Ji, in Proceedings of IEEE International Conference on Computer Vision (ICCV). Removing rain from a single image via discriminative sparse coding (2015), pp. 3397–3405

  24. Y. Li, R.T. Tan, X.J. Guo, J.B. Lu, M.S. Brown, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Rain streak removal using layer priors (2016), pp. 2736–2744

  25. Y. Li, R.T. Tan, X.J. Guo, J.B. Lu, M.S. Brown, Single image rain streak decomposition using layer priors. IEEE Trans. Image Process. 26(8), 3874–3885 (2017)

    Article  MathSciNet  Google Scholar 

  26. J.C. Li, F.M. Fang, K.F. Mei, G.X. Zhang, in Proceedings of Conference on European Conference on Computer Vision (ECCV). Multi-scale residual network for image super-resolution (2018), pp. 517–532

  27. M.H. Li, Q. Xie, Q. Zhao, W. Wei. S.H. Gu, J. Tao, D. Meng, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Video rain streak removal by multiscale convolutional sparse coding (2018), pp. 6644–6653

  28. A.L. Maas, A.Y. Hannun, A.Y. Ng, in International Conference on Machine Learning (ICML). Rectififier nonlinearities improve neural network acoustic models (2013)

  29. J. Pan, S. Liu, D. Sun, J. Zhang, Y. Liu, J. Ren, Z. Li, J. Tang, H. Lu, Y. Tai, M. Yang, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Learning dual convolutional neural networks for low-level vision (2018), pp. 3070–3079

  30. H. Prasad, N. Rashmika, A. Damminda, D.S. Daswin, C. Naveen, Rain streak removal for single images using conditional generative adversarial networks. Appl. Sci. 11(5), 2214 (2021)

    Article  Google Scholar 

  31. C. Tian, Y. Xu, W. Zuo, Image denoising using deep CNN with batch renormalization. Neural Netw. 121, 461–473 (2020)

    Article  Google Scholar 

  32. W. Wei, L.X. Yi, Q. Xie, Q. Zhao, D.Y. Meng, Z.B. Xu, in Proceedings of IEEE International Conference on Computer Vision (ICCV). Should we encode rain streaks in video as deterministic or stochastic (2017), pp. 2516–2525

  33. J.S. Wang, X. Huang, S. Gai, Singel image rain removal via cascading attention aggregation network on challenging weather conditions. IEEE Access 7, 178848–178861 (2019)

    Article  Google Scholar 

  34. Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  35. J.Y. Xie, L.L. Xu, E.H. Chen, in Proceedings of Neural Information Processing Systems (NIPS). Image denoising and inpainting with deep neural networks (2012), pp. 131–140

  36. W.H. Yeo, Y.J. Heo, Y.J. Choi, B.G. Kim, Place classification algorithm based on semantic segmented objects. Appl. Sci. 10(24), 9069 (2020)

    Article  Google Scholar 

  37. W.H. Yang, R.T. Tan, J.S. Feng, J.Y. Liu, Z.M. Guo, S.C. Yan, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Deep joint rain detection and removal from a single image (2017), pp. 1357–1366

  38. F. Yu, V. Koltun, in international conference for learning representations (ICLR). Multi-scale context aggregation by dilated convolutions (2016)

  39. L. Zhu, C.W. Fu, D. Lischinski, P.A. Heng, in Proceedings of IEEE International Conference on Computer Vision (ICCV). Joint bi-layer optimization for single-image rain streak removal (2017), pp. 2526–2534

  40. H. Zhang, V.M. Patel, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Density-aware single image de-raining using a multi-stream dense network (2018), pp. 695–704

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 62061032, 61866027, the Natural Science Foundation of Youth Key Project of Jiangxi Province under Grant 20192ACB21032 and the Jiangxi Science Foundation under Grant 20202BABL202038.

Author information

Authors and Affiliations

  1. School of Information Engineering, Nanchang Hangkong University, Nanchang, Jiangxi, 330063, People’s Republic of China

    Qiangqiang Xie, Hai Zhang, Shan Gai & Bangshu Xiong

Authors
  1. Qiangqiang Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shan Gai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bangshu Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shan Gai.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, Q., Zhang, H., Gai, S. et al. New Single Image Rain Removal Algorithm Based on Dual Parallel Branch Residual Overlay Network. Circuits Syst Signal Process 41, 2188–2204 (2022). https://doi.org/10.1007/s00034-021-01883-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-021-01883-7

Keywords

Search

Navigation