Skip to main content
SpringerLink
Log in

Single-image rain removal using deep residual network

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Rain not only reduces visibility, but also prevents many computer vision algorithms from functioning properly. Removing rain from a single image is an important issue in the field of computer vision. Rain usually produces a phenomenon similar to fog, which is more pronounced when the rain is dense. We constructed a rain model, including rain streaks and fog generated by rainfall. Based on this model, we constructed a multi-task learning network to learn the rain streaks and fog in the image as well as the final clean background. To take advantage of contextual information, we use an expanded convolution network to handle different levels of rain and fog. We learn the residual images of fog and rain based on the deep residual network to avoid the influence of inaccurate parameter estimation on the results. Experiments show that the method in this paper can effectively remove rain and fog in the image.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Shen, L., Yue, Z., Chen, Q., Feng, F., Ma, J.: Deep joint rain and haze removal from a single image. In: IEEE International Conference on Pattern Recognition, pp. 2821–2826 (2018)

  2. Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: European Conference on Computer Vision, pp. 254–269 (2018)

  3. Du, S., Liu, Y., Ye, M., Xu, Z., Li, J., Liu, J.: Single image deraining via decorrelating the rain streaks and background scene in gradient domain. Pattern Recognit. 79, 303–317 (2018)

    Article  Google Scholar 

  4. Wang, X., Chen, J., Jiang, K., Han, Z., Ruan, W., Wang, Z., Liang, C.: Single image de-raining via clique recursive feedback mechanism. Neurocomputing 417, 142–154 (2020)

    Article  Google Scholar 

  5. Yi, P., Wang, Z., Jiang, K., Shao, Z., Ma, J.: Multi-temporal ultra dense memory network for video super-resolution. IEEE Trans. Circuits Syst. Video Technol. 30(8), 2503–2516 (2020)

    Article  Google Scholar 

  6. Pal, N.S., Lal, S., Shinghal, K.: A robust visibility restoration framework for rainy weather degraded images. TEM J. 7(4), 859–868 (2018)

    Google Scholar 

  7. Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, JJIToIP: Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26(6), 2944–2956 (2017)

    Article  MathSciNet  Google Scholar 

  8. Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: European Conference on Computer Vision, pp. 262-277 (2018)

  9. Jiang, K., Wang, Z., Yi, P., Chen, C., Huanng, B., Luo, Y., et al.: Multi-scale progressive fusion network for single image deraining. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 8343–8352 (2020)

  10. Ren, D., Zuo, W., Hu, Q., Zhu, P., Meng, D.: Progressive image deraining networks: a better and simpler baseline. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3932–3941 (2019)

  11. Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., Yan, S.: Deep joint rain detection and removal from a single image. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1357–1366 (2017)

  12. Huang, D.-A., Kang, L.-W., Yang, M.-C., Lin, C.-W., Wang, Y.-C.F.: Context-aware single image rain removal. In: IEEE International Conference on Multimedia and Expo, pp. 164–169 (2012)

  13. Chen, D.-Y., Chen, C.-C., Kang, L.-W.: Visual depth guided color image rain streaks removal using sparse coding. Circuits Syst. Video Technol. 24(8), 1430–1455 (2014)

    Article  Google Scholar 

  14. Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3397–3405 (2015)

  15. Aceto, G., Ciuonzo, D., Montieri, A., Pescapé, A.: Toward effective mobile encrypted traffic classification through deep learning. Neurocomputing 409, 306–315 (2020)

    Article  Google Scholar 

  16. Ngiam, J., Khosla, A., Kim, M., et al.: Multimodal deep learning, machine learning, ICML pp. 689-696 (2011)

  17. Aceto, G., Ciuonzo, D., Montieri, A., Pescapè, A.: MIMETIC: Mobile encrypted traffic classification using multimodal deep learning. Comput. Netw. 165, 106944 (2019)

    Article  Google Scholar 

  18. Li, J., Li, G., Fan, H.: Image dehazing using residual-based deep CNN. IEEE Access 6, 26831–26842 (2018)

    Article  Google Scholar 

  19. He, K., Sun, J., Fellow et al: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2011)

    Article  Google Scholar 

  20. Liu, D., Wen, B., Liu, X., Wang, Z., Huang, T.: When image denoising meets high-level vision tasks: a deep learning approach. In: International Joint Conference on Artificial Intelligence, pp. 842–848 (2018)

  21. Li, R., Cheong, L.F., Tan, R.: Single image deraining using scale-aware multi-stage recurrent network, arXiv:1712.06830, (2017)

  22. Fan, Z., Wu, H., Fu, X., Huang, Y., Ding, X.: Residual-guide network for single image deraining. In: ACM Multimedia Conference on Multimedia Conference, pp. 1751–1759 (2018)

  23. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  24. Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1715–1723 (2017)

  25. Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: IEEE International Conference on Computer Vision, pp. 3397–3405 (2015)

  26. Li, Y., Tan, R.T., Guo, X., Lu, J., Brown, M.S.: Rain streak removal using layer priors. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2736–2744 (2016)

  27. Zhang, H., Sindagi, V., Patel, V.: Image de-raining using a conditional generative adversarial network. In: IEEE Transactions on Circuits and Systems for Video Technology, Early access, https://doi.org/10.1109/TCSVT.2019.2920407

  28. Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: European Conference on Computer Vision, pp. 746–760 (2012)

  29. Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. In: ACM International Conference on Multimedia, pp. 675–678 (2014)

  30. Huynh-Thu, Q., Ghanbari, M.: Scope of validity of PSNR in image/video quality assessment. Electron. Lett. 44(13), 800–801 (2008)

    Article  Google Scholar 

  31. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  32. Sheikh, H.R., Bovik, A.C.: Image information and visual quality. IEEE Trans. Image Process. 15(2), 430–444 (2006)

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (61772319, 61976125, 61976124, 61907026).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Shandong Technology and Business University, Yantai, 264005, China

    Genji Yuan, Jinjiang Li & Zhen Hua

  2. Co-innovation Center of Shandong Colleges and Universities: Future Intelligent Computing, Yantai, 264005, China

    Genji Yuan & Jinjiang Li

Authors
  1. Genji Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinjiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhen Hua
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jinjiang Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, G., Li, J. & Hua, Z. Single-image rain removal using deep residual network. SIViP 15, 827–834 (2021). https://doi.org/10.1007/s11760-020-01803-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-020-01803-3

Keywords

Search

Navigation