Skip to main content

Advertisement

Springer Nature Link
Log in

A two-stage-UNet network based on group normalization for single image deraining

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Rain streaks can seriously damage the optical quality of image and affect image processing in many scenes. Deep learning methods achieve state-of-the-art performance in the task of single-image rain removal. However, most deraining models based on deep learning only deal with local relationships, they didn’t sufficiently consider the contextual information over long distances in the task of rain removal. This drawback can lead to residual rain streaks and insufficient recovery of texture details. Therefore, a Two-Stage-UNet Network based on Group Normalization named TSUGN is created to solve these problems. It decomposes the image deraining task into easier and smaller subtasks for capturing more contextual information. And in order to balance spatial details and high-level contextual information, group normalization is also added to our Group Normalization Feature Residual Block (GNFRB). By fully taking into account of multi-scale features information, a Scale-Feature Fusion Module(SFFM)is proposed to learn features with different scales. In addition, a new feature compensation method is proposed to deal with the model bias issue by combining a parameter-free \(3-D\) attention module SimAM with GNFRB. Comprehensive experiments demonstrate the superiority of the proposed network in terms of computational efficiency, end-to-end trainability and easy implementation. It has great potential in image recovery tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The data supporting this study’s findings are available from the corresponding author upon reasonable request.

References

  1. Chen, J., Tan, C.-H., Hou, J., Chau, L.-P., Li, H.: Robust video content alignment and compensation for rain removal in a cnn framework. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6286–6295 (2018)

  2. Zhou, W., Wang, X.: Twin-stage unet-like network for single image deraining. Signal, Image and Video Processing, 1–9 (2023)

  3. Chen, D., Chen, C., Kang, L.: Visual depth guided color image rain streaks removal using sparse coding, pp. 1430–1455 (2014)

  4. Zhang, H., Patel, V.M.: Convolutional sparse and low-rank coding-based rain streak removal. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1259–1267 (2017)

  5. Li, Y., Tan, R.T., Guo, X., Lu, J., Brown, M.S.: Rain streak removal using layer priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2736–2744 (2016)

  6. Hautière, N., Tarel, J.-P., Aubert, D.: Towards fog-free in-vehicle vision systems through contrast restoration. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

  7. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior, pp. 2341–2353 (2010)

  8. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks, pp. 295–307 (2015)

  9. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising, pp. 3142–3155 (2016)

  10. Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3855–3863 (2017)

  11. Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., Yan, S.: Deep joint rain detection and removal from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1357–1366 (2017)

  12. Li, G., He, X., Zhang, W., Chang, H., Dong, L., Lin, L.: Non-locally enhanced encoder-decoder network for single image de-raining. In: Proceedings of the 26th ACM International Conference on Multimedia, pp. 1056–1064 (2018)

  13. Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network, pp. 3943–3956 (2019)

  14. Ren, D., Zuo, W., Hu, Q., Zhu, P., Meng, D.: Progressive image deraining networks: A better and simpler baseline. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3937–3946 (2019)

  15. Mutual channel prior guided dual-domain interaction network for single image raindrop removal, pp. 132–142 (2023)

  16. Liu, Y., Zhang, R., Zhang, Y., Yao, X., Han, H.: Single image deraining via a recurrent multi-attention enhancement network. Signal Processing: Image Communication, 116926 (2023)

  17. Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.-H., Shao, L.: Multi-stage progressive image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14821–14831 (2021)

  18. Zou, W., Wang, Y., Fu, X., Cao, Y.: Dreaming to prune image deraining networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6023–6032 (2022)

  19. Wu, Y., He, K.: Group normalization. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  20. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation, pp. 234–241 (2015)

  21. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

  22. Yang, L., Zhang, R.-Y., Li, L., Xie, X.: Simam: A simple, parameter-free attention module for convolutional neural networks. In: International Conference on Machine Learning, pp. 11863–11874 (2021)

  23. Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift, pp. 448–456 (2015)

  24. Tian, Z., Shen, C., Chen, H., He, T.: Fcos: Fully convolutional one-stage object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9627–9636 (2019)

  25. Zhao, H., Zhang, Y., Liu, S., Shi, J., Loy, C.C., Lin, D., Jia, J.: Psanet: Point-wise spatial attention network for scene parsing. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 267–283 (2018)

  26. Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 254–269 (2018)

  27. Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: Cbam: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  28. Nanba, Y., Miyata, H., Han, X.-H.: Dual heterogeneous complementary networks for single image deraining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 568–577 (2022)

  29. Li, P., Jin, J., Jin, G., Fan, L., Gao, X., Song, T., Chen, X.: Deep scale-space mining network for single image deraining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4276–4285 (2022)

  30. Chen, X., Pan, J., Jiang, K., Li, Y., Huang, Y., Kong, C., Dai, L., Fan, Z.: Unpaired deep image deraining using dual contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2017–2026 (2022)

  31. Chen, L., Lu, X., Zhang, J., Chu, X., Chen, C.: Hinet: Half instance normalization network for image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 182–192 (2021)

  32. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity, pp. 600–612 (2004)

  33. Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3855–3863 (2017)

  34. Huynh-Thu, Q., Ghanbari, M.: Scope of validity of psnr in image/video quality assessment, pp. 800–801 (2008)

  35. Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network, pp. 3943–3956 (2019)

  36. Zhang, H., Patel, V.M.: Density-aware single image de-raining using a multi-stream dense network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 695–704 (2018)

  37. Wang, T., Yang, X., Xu, K., Chen, S., Zhang, Q., Lau, R.W.: Spatial attentive single-image deraining with a high quality real rain dataset, pp. 12270–12279 (2019)

  38. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  39. Yang, D., He, X., Zhang, R.: Alternating attention transformer for single image deraining. Digital Signal Processing, 104144 (2023)

Download references

Author information

Authors and Affiliations

  1. College of Information Engineering, Shanghai Maritime University, Shanghai, 201306, China

    Weina Zhou & Hao Han

Authors
  1. Weina Zhou
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Hao Han
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Weina Zhou: Conceptualization, Methodology, Resources, Supervision, Writing – review & editing, Project administration. Hao Han: Methodology, Software, Validation, Formal analysis, Investigation, Data curation, Writing – original draft, Visualization.

Corresponding author

Correspondence to Weina Zhou.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Communicated by Q. Xu.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, W., Han, H. A two-stage-UNet network based on group normalization for single image deraining. Multimedia Systems 30, 184 (2024). https://doi.org/10.1007/s00530-024-01362-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00530-024-01362-4

Keywords