Abstract
Most existing deraining methods use synthetic rainy images to train models. They focus on extracting the features to establish mapping models from rainy images to clean background images, while ignoring the domain gap between synthetic and real rainy images. Further more, the rain streaks generation is not considered as important as the rain removal. Hence, we propose a novel Generating-Removing United Unpaired image deraining Network(GRUUNet) to generate and remove rain streaks unitedly. It helps to reduce the difference between synthetic and real rain streaks to improve the performance of deraining. Specifically, (1)we adopt a dual-way transform strategy between real and synthetic rainy images. There are both rain streaks generation and removal network for real and synthetic rainy images respectively; (2)our model learns the prototypes of rainy degradation images by self-supervised sparse-addressing memory modules; (3)we align the domain gap between real and synthetic rainy images with shared rain streaks generation network. We empirically evaluate our GRUUNet on Cityscape and RainHQ, and the outstanding results prove the promising performance of our method on both real and synthetic rainy images.
Y. Chen and Z. Yan—These authors contributed equally to this work and share first authorship.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, J., et al.: Robust video content alignment and compensation for rain removal in a CNN framework. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Fu, X., et al.: Removing rain from single images via a deep detail network. In: IEEE Conference on Computer Vision & Pattern Recognition (2017)
Garg, K., Nayar, S.K.: Vision and rain. Int. J. Comput. Vis. 75, 3–27 (2007)
Gong, D., et al.: Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (2020)
Hu, X., et al.: Depth-attentional features for single-image rain removal. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Huang, H., Yu, A., He, R.: Memory oriented transfer learning for semi-supervised image deraining. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7732–7741 (2021)
Li, X., et al.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Proceedings of the European conference on computer vision (ECCV), pp. 254–269 (2018)
Li, Y., et al.: Rain streak removal using layer priors. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2736–2744 (2016)
Lin, H., et al.: Rain O’er Me: synthesizing real rain to derain with data distillation. IEEE Trans. Image Process. 29, 7668–7680 (2020). https://doi.org/10.1109/TIP.2020.3005517
Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3397–3405 (2015)
Qian, R., et al.: Attentive generative adversarial network for raindrop removal from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2482–2491 (2018)
Sukanta Halder, S., Lalonde, J.-F., de Charette, R.: Physics-based rendering for improving robustness to rain. In: arXiv e-prints, pp. arXiv-1908 (2019)
Sukhbaatar, S., et al.: End-to-end memory networks. In: Advances in Neural Information Processing Systems 28 (2015)
Wang, T., et al.: Spatial attentive single-image deraining with a high quality real rain dataset. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12270–12279 (2019)
Wei, W., et al.: Semi-supervised transfer learning for image rain removal. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Weston, J., Chopra, S., Bordes, A.: Memory networks. In: arXiv preprint arXiv:1410.3916 (2014)
Yang, W., et al.: Deep joint rain detection and removal from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1357–1366 (2017)
Yasarla, R., Sindagi, V.A., Patel, V.M.: Syn2Real transfer learning for image deraining using gaussian processes. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2723–2733 (2020)
Ye, Y., et al.: Closing the loop: joint rain generation and removal via disentangled image translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2053–2062 (2021)
Ye, Y., et al.: Unsupervised Deraining: where contrastive learning meets self-similarity. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5821–5830 (2022)
Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. In: IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, issue 11, pp. 3943–3956 (2019)
Acknowledgment
This work was supported in part by the National Key R &D Program of China (No. 2021YFA1003004), in part by the Shanghai Municipal Natural Science Foundation (No.21ZR1423300).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Chen, Y., Yan, Z., Ma, L. (2024). New Insights on the Generation of Rain Streaks: Generating-Removing United Unpaired Image Deraining Network. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14435. Springer, Singapore. https://doi.org/10.1007/978-981-99-8552-4_31
Download citation
DOI: https://doi.org/10.1007/978-981-99-8552-4_31
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8551-7
Online ISBN: 978-981-99-8552-4
eBook Packages: Computer ScienceComputer Science (R0)