Skip to main content
SpringerLink
Log in

Densely connected multi-scale de-raining net

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Rainy images severely degrade the visibility and make many computer vision algorithms invalid. Hence, it is necessary to remove rain streaks from single image. In this paper, we propose a novel network to handle with single image de-raining, which includes two modules: (a) multi-scale kernels de-raining layer and (b) multi-scale feature maps de-raining layer. Specifically, as spatial contextual information is important for single image de-raining, we develop a multi-scale kernels de-raining layer, which can utilize the multi-scale kernel that has receptive fields with different sizes to further capture the contextual information and these features are fused to learn the primary rain streaks structures. Moreover, we illustrate that convolution layers at different scales have similar structure of rain streaks by statistical pixel histogram and they can be processed in the same operation. So, we deal with the rain streaks information at different scales by using multi-scale kernels de-raining layers with shared parameters, where we call this operation as multi-scale feature maps de-raining layer. Finally, we employ dense connections to connect multi-scale feature maps de-raining layers to maximize the information flow along features from different levels. Quantitative and qualitative experimental results demonstrate the superiority of proposed method compared with several state-of-the-art de-raining methods, while the parameters of our proposed method are greatly reduced that benefits from the proposed shared parameters strategy at different scales

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Brewer N, Liu N (2008) Using the shape characteristics of rain to identify and remove rain from video. In: Structural, syntactic, and statistical pattern recognition, pp. 451–458. https://doi.org/10.1007/978-3-540-89689-0_49

  2. Cai B, Xu X, Jia K, Qing C, Tao D (2016) Dehazenet: An end-to-end system for single image haze removal. In: IEEE Transactions on Image Processing. https://doi.org/10.1109/TIP.2016.2598681, vol 25, pp 5187–5198

  3. Chen Y, Hsu C (2013) A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In: ICCV, pp. 1968–1975. https://doi.org/10.1109/ICCV.2013.247

  4. Cui Z, Chang H, Shan S, Zhong B, Chen X (2014) Deep network cascade for image super-resolution. In: ECCV, pp. 49–64. https://doi.org/10.1007/978-3-319-10602-1_4

  5. Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. In: IEEE Trans Pattern Anal Mach Intell, vol 38, pp 295–307. https://doi.org/10.1109/TPAMI.2015.2439281

  6. Fu X, Huang J, Ding X, Liao Y, Paisley J (2017) Clearing the skies: A deep network architecture for single-image rain removal. In: IEEE Transactions on Image Processing, vol 26, pp 2944–2956. https://doi.org/10.1109/TIP.2017.2691802

  7. Fu X, Huang J, Zeng D, Huang Y, Ding X, Paisley J, Removing rain from single images via a deep detail network. In: CVPR pp. 1715–1723 (2017). https://doi.org/10.1109/CVPR.2017.186

  8. Garg K, Nayar SK (2004) Detection and removal of rain from videos. In: CVPR, pp. 528–535. https://doi.org/10.1109/CVPR.2004.79

  9. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: CVPR, pp. 7132–7141. https://doi.org/10.1109/CVPR.2018.00745, http://openaccess.thecvf.com/content_cvpr_2018/html/H_Squeeze-and-Excitation_NetworksCVPR_2018_paper.html

  10. Huang D, Kang L, Yang M, Lin C, Wang YF (2012) Context-aware single image rain removal. In: Proceedings of the 2012 IEEE International Conference on Multimedia and Expo, ICME 2012, Melbourne, Australia, July 9-13, 2012, pp. 164–169 . https://doi.org/10.1109/ICME.2012.92

  11. Huang G, Liu Z, van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: CVPR, pp. 2261–2269 . https://doi.org/10.1109/CVPR.2017.243

  12. Huynh-Thu Q, Ghanbari M (2008) Scope of validity of psnr in image/video quality assessment. Electron Lett 44(13):800–801

    Google Scholar 

  13. Kang L, Lin C, Fu Y (2012) Automatic single-image-based rain streaks removal via image decomposition. In: IEEE Transactions on Image Processing, vol 21, pp 1742–1755. https://doi.org/10.1109/TIP.2011.2179057

  14. Kim J, Lee C, Sim J, Kim C (2013) Single-image deraining using an adaptive nonlocal means filter. In: ICIP, pp. 914–917. https://doi.org/10.1109/ICIP.2013.6738189

  15. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. In: CoRR, arXiv:1412.6980

  16. Li Y, Tan RT, Guo X, Lu J, Brown MS (2016) Rain streak removal using layer priors. In: CVPR, pp. 2736–2744. https://doi.org/10.1109/CVPR.2016.299

  17. Li B, Peng X, Wang Z, Xu J, Feng D (2017) Aod-net: All-in-one dehazing network. In: ICCV, pp. 4780–4788. https://doi.org/10.1109/ICCV.2017.511

  18. Li G, He X, Zhang W, Chang H, Dong L, Lin L (2018) Non-locally enhanced encoder-decoder network for single image de-raining. In: ACM MM, pp. 1056–1064. https://doi.org/10.1145/3240508.3240636

  19. Li X, Wu J, Lin Z, Liu H, Zha H (2018) Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: ECCV, pp. 262–277. https://doi.org/10.1007/978-3-030-01234-2_16

  20. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015, pp. 3431–3440. https://doi.org/10.1109/CVPR.2015.7298965

  21. Luo Y, Xu Y, Ji H (2015) Removing rain from a single image via discriminative sparse coding. In: ICCV, pp. 3397–3405. https://doi.org/10.1109/ICCV.2015.388

  22. Memos VA, Psannis KE, Ishibashi Y, Kim B, Gupta BB (2018) An efficient algorithm for media-based surveillance system (eamsus) in iot smart city framework. Future Generation Comp Syst 83:619–628. https://doi.org/10.1016/j.future.2017.04.039

    Google Scholar 

  23. Mustaniemi J, Kannala J, Särkkä S, Matas J, Heikkilä J (2018) Inertial-aided motion deblurring with deep networks. In: CoRR, arXiv:1810.00986

  24. Plageras AP, Psannis KE, Stergiou C, Wang H, Gupta BB (2018) Efficient iot-based sensor BIG data collection-processing and analysis in smart buildings. Future Generation Comp Syst 82:349–357. https://doi.org/10.1016/j.future.2017.09.082

    Google Scholar 

  25. Psannis KE, Ishibashi Y (2006) Impact of video coding on delay and jitter in 3g wireless video multicast services. EURASIP J. Wireless Comm. and Networking 2006. https://doi.org/10.1155/WCN/2006/24616

  26. Qian R, Tan RT, Yang W, Su J, Liu J (2018) Attentive generative adversarial network for raindrop removal from a single image. In: CVPR, pp. 2482–2491. https://doi.org/10.1109/CVPR.2018.00263, http://openaccess.thecvf.com/content_cvpr2018/html/Qian_Attentive_Generative_Adversarial_CVPR_2018_paper.html

  27. Ren W, Liu S, Zhang H, Pan J, Cao X, Yang M (2016) Single image dehazing via multi-scale convolutional neural networks. In: ECCV, pp. 154–169. https://doi.org/10.1007/978-3-319-46475-6_10

  28. Ren D, Zuo W, Hu Q, Zhu P, Meng D (2019) Progressive image deraining networks: a better and simpler baseline. IN: CVPR

  29. Santhaseelan V, Asari VK (2015) Utilizing local phase information to remove rain from video. pp. 71–89 . https://doi.org/10.1007/s11263-014-0759-8

  30. Stergiou C, Psannis K, Plageras A, Ishibashi Y, Kim BG (2018) Algorithms for efficient digital media transmission over iot and cloud networking. EURASIP J. Wireless Comm. and Networking 5(1):1–10. https://doi.org/10.1155/WCN/2006/24616

    Google Scholar 

  31. Stergiou C, Psannis KE, Kim B, Gupta BB (2018) Secure integration of iot and cloud computing. Future Generation Comp Syst 78:964–975. https://doi.org/10.1016/j.future.2016.11.031

    Google Scholar 

  32. Tripathi AK, Mukhopadhyay S (2014) Removal of rain from videos: a review. pp. 1421–1430. https://doi.org/10.1007/s11760-012-0373-6

  33. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Processing 13(4):600–612. https://doi.org/10.1109/TIP.2003.819861

    Google Scholar 

  34. Wang X, Shrivastava A, Gupta A (2017) A-fast-rcnn: Hard positive generation via adversary for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pp. 3039–3048. https://doi.org/10.1109/CVPR.2017.324

  35. Wang T, Yang X, Xu K, Chen S, Zhang Q, Lau RW (2019) Spatial attentive single-image deraining with a high quality real rain dataset. In: CVPR

  36. Yang W, Tan RT, Feng J, Liu J, Guo Z, Yan S (2017) Deep joint rain detection and removal from a single image. In: CVPR, pp. 1685–1694. https://doi.org/10.1109/CVPR.2017.183

  37. Yu J, Fan Y, Yang J, Xu N, Wang Z, Wang X, Huang T (2018) Wide activation for efficient and accurate image super-resolution. arXiv:1808.08718

  38. Zhang H, Sindagi V, Patel VM (2017) Image de-raining using a conditional generative adversarial network. In: CoRR, arxiV:1701.05957

  39. Zhang H, Patel VM (2018) Densely connected pyramid dehazing network. In: CVPR, pp. 3194–3203. https://doi.org/10.1109/CVPR.2018.00337, http://openaccess.thecvf.com/content_cvpr_2018/html/Zhang_Densely_Connected_Pyramid_CVPR_2018_paper.html

  40. Zhang H, Patel VM (2018) Density-aware single image de-raining using a multi-stream dense network. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 695–704. https://doi.org/10.1109/CVPR.2018.00079, http://openaccess.thecvf.com/content_cvpr_2018/html/Zhang_Density-Aware_Single_Image_CVPR_2018_paper.html

  41. Zhang Y, Wang L, Qi J, Wang D, Feng M, Lu H (2018) Structured siamese network for real-time visual tracking. In: Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part IX, pp. 355–370. https://doi.org/10.1007/978-3-030-01240-3_22

  42. Zhenghao Shi Yaning Feng MZLH (2019) A joint deep neural networks-based method for single nighttime rainy image enhancement Neural Computing and Applications

Download references

Acknowledgements

This work was supported by the Natural Science Foundation of China [grant numbers 61572099]; Major National Science and Technology Project of China [grant number 2018ZX04011001-007].

Author information

Authors and Affiliations

  1. Dalian University of Technology, Dalian, China

    Cong Wang, Man Zhang & Zhixun Su

  2. Guilin University of Electronic Technology, Guilin, China

    Zhixun Su

  3. Chengdu University of Technology, Chengdu, China

    Guangle Yao

Authors
  1. Cong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Man Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhixun Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guangle Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cong Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, C., Zhang, M., Su, Z. et al. Densely connected multi-scale de-raining net. Multimed Tools Appl 79, 19595–19614 (2020). https://doi.org/10.1007/s11042-020-08855-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08855-0

Keywords

Search

Navigation