Abstract
In recent years, single image deraining has received considerable research interests. Supervised learning is widely adopted for training dedicated deraining networks to achieve promising results on synthetic datasets, while limiting in handling real-world rainy images. Unsupervised and semi-supervised learning-based deranining methods have been studied to improve the performance on real cases, but their quantitative results are still inferior. In this paper, we propose to address this crucial issue for image deraining in terms of backbone architecture and the strategy of semi-supervised learning. First, in terms of network architecture, we propose an attentive image deraining network (AIDNet), where residual attention block is proposed to exploit the beneficial deep feature from the rain streak layer to background image layer. Then, different from the traditional semi-supervised method by enforcing the consistency of rain pattern distribution between real rainy images and synthetic rainy images, we explore the correlation between the real clean images and the predicted background image by imposing adversarial losses in wavelet space \(I _{HH}\), \(I _{HL}\), and \(I _{LH}\), resulting in the final AID-DWT model. Extensive experiments on both synthetic and real-world rainy images have validated that our AID-DWT can achieve better deraining results than not only existing semi-supervised deraining methods qualitatively but also outperform state-of-the-art supervised deraining methods quantitatively. All the source code and pre-trained models are available at https://github.com/cuiyixin555/DeRain-DWT.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen, Y., Hsu, C.: A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In: IEEE ICCV (2013)
Deng, S., et al.: Detail-recovery image deraining via context aggregation networks. In: IEEE CVPR, pp. 14548–14557 (2020)
Fan, Z., Wu, H., Fu, X., Hunag, Y., Ding, X.: Residual-guide feature fusion network for single image deraining. In: ACM MM, pp. 1751–1759 (2018)
Fu, X., Liang, B., Huang, Y., Ding, X., Paisley, J.: Lightweight pyramid networks for image deraining. In: IEEE TNNLS (2020)
Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, J.: Clearing the skies: a deep network architecture for single-image rain removal. IEEE TIP 26(6), 2944–2956 (2017)
Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: IEEE CVPR, pp. 1715–1723 (2017)
Fu, X., Liang, B., Huang, Y., Ding, X., Paisley, J.: Lightweight pyramid networks for image deraining. IEEE TNNLS 31(6), 1–14 (2019)
Chen, H., et al.: Pre-trained image processing transformer. arXiv:2012.00364 (2021)
Huynh-Thu, Q., Ghanbari, M.: Scope of validity of PSNR in image/video quality assessment. Electron. Lett. 44(13), 800–801 (2008)
Jiang, K., et al.: Multi-scale progressive fusion network for single image deraining. In: IEEE CVPR, pp. 8343–8352 (2020)
Kang, L., Lin, C., Fu, Y.: Automatic single-image-based rain streaks removal via image decomposition. In: IEEE TIP (2012)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)
Li, G., He, X., Zhang, W., Chang, H., Dong, L., Lin, L.: Non-locally enhanced encoder-decoder network for single image de-raining. In: ACM MM, pp. 1056–1064 (2018)
Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 262–277. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_16
Li, Y., Tan, R.T., Guo, X., Lu, J., Brown, M.S.: Rain streak removal using layer priors. In: IEEE CVPR, pp. 2736–2744 (2016)
Li, Y., Tan, R.T., Guo, X., Lu, J., Brown, M.S.: Rain streak removal using layer priors (2016)
Liu, J.J., Hou, Q., Cheng, M.M., Wang, C., Feng, J.: Improving convolutional networks with self-calibrated convolutions. In: IEEE CVPR, pp. 10093–10102 (2020)
Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: IEEE ICCV (2015)
Mallat, S.G.: A theory for multiresolution signal decomposition: the wavelet representation. IEEE TPAMI 11(7), 674–693 (1989)
Mittal, A., Soundararajan, R., Bovik, A.C.: Making a “completely blind’’ Image quality analyzer. IEEE Sig. Process. Lett. 20, 209–212 (2013)
Pan, J., Hu, Z., Su, Z., Yang, M.: \(l_0\)-regularized intensity and gradient prior for deblurring text images and beyond. In: IEEE TPAMI (2017)
Pang, B., Zhai, D., Jiang, J., Liu, X.: Single image deraining via scale-space invariant attention neural network. In: ACM MM, pp. 375–383 (2020)
Paszke, A., et al.: Automatic differentiation in pytorch. In: NIPS Autodiff Workshop The Future of Gradient-based Machine Learning Software and Techniques
Ren, D., Shang, W., Zhu, P., Hu, Q., Meng, D., Zuo, W.: Single image deraining using bilateral recurrent network. IEEE TIP 29, 6852–6863 (2020)
Ren, D., Zuo, W., Zhang, D., Zhang, L., Yang, M.: Simultaneous fidelity and regularization learning for image restoration. IEEE TPAMI 43(1), 284–299 (2019)
Ren, D., Zuo, W., Hu, Q., Zhu, P., Meng, D.: Progressive image deraining networks: a better and simpler baseline. In: IEEE CVPR (2019)
Wang, T., et al.: Spatial attentive single-image deraining with a high quality real rain dataset. In: IEEE CVPR (2019)
Yang, W., et al.: Deep joint rain detection and removal from a single image. In: IEEE CVPR, pp. 1357–1366 (2017)
Wang, C., Wu, Y., Su, Z., Chen, J.: Joint self-attention and scale-aggregation for self-calibrated deraining network. In: ACM MM, pp. 2517–2525 (2019)
Wang, C., Xing, X., Wu, Y., Su, Z., Chen, J.: DCSFN: deep cross-scale fusion network for single image rain removal. In: ACM MM, pp. 1643–1651 (2019)
Wei, W., Meng, D., Zhao, Q., Xu, Z., Wu, Y.: Semi-supervised transfer learning for image rain removal. In: IEEE CVPR, pp. 3872–3881 (2019)
Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: CBAM: convolutional block attention module. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 3–19. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_1
Li, X., et al.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: ECCV, pp. 2736–2744 (2016)
Yang, Y., Lu, H.: Single image deraining via recurrent hierarchy enhancement network. In: ACM MM, pp. 1814–1822 (2019)
Yasarla, R., Sindagi, V.A., Patel, V.M.: Syn2real transfer learning for image deraining using gaussian processes. In: IEEE CVPR, pp. 2723–2733 (2020)
Dai, Y., Gieseke, F., Oehmcke, S., Wu, Y., Barnard, K.: Attentional feature fusion. In: IEEE WACV (2021)
Zhang, H., Patel, V.M.: Density-aware single image de-raining using a multi-stream dense network. In: IEEE CVPR, pp. 695–704 (2018)
Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. IEEE TCSVT 30(11), 3943–3956 (2019)
Zhao, H., Jia, J., Koltun, V.: Exploring self-attention for image recognition. In: IEEE CVPR (2020)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE TIP 13(4), 600–612 (2004)
Zhu, H., Wang, C., Zhang, Y., Su, Z., Zhao, G.: Physical model guided deep image deraining. In: IEEE ICME, pp. 1–6 (2020)
Zhu, H., et al.: Singe image rain removal with unpaired information: a differentiable programming perspective. In: AAAI, pp. 9332–9339 (2019)
Zhu, J., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE ICCV, pp. 2242–2251 (2017)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Cui, X., Shang, W., Ren, D., Zhu, P., Gao, Y. (2021). Semi-supervised Single Image Deraining with Discrete Wavelet Transform. In: Pham, D.N., Theeramunkong, T., Governatori, G., Liu, F. (eds) PRICAI 2021: Trends in Artificial Intelligence. PRICAI 2021. Lecture Notes in Computer Science(), vol 13033. Springer, Cham. https://doi.org/10.1007/978-3-030-89370-5_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-89370-5_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89369-9
Online ISBN: 978-3-030-89370-5
eBook Packages: Computer ScienceComputer Science (R0)