Abstract
Single image deraining is an urgent yet challenging task since rain streaks severely degrade the image quality and hamper the practical application. The investigation on rain removal has thus been attracting, while the performances of existing deraining have limitations owing to over smoothing effect, poor generalization capability and rain intensity varies both in spatial locations and color channels. To address these issues, we proposed a Multi-scale Attentive Residual Dense Network called MARD-Net in end-to-end manner, to exactly extract the negative rain streaks from rainy images while precisely preserving the image details. The architecture of modified dense network can be used to exploit the rain streaks details representation through feature reuse and propagation. Further, the Multi-scale Attentive Residual Block (MARB) is involved in the dense network to guide the rain streaks feature extraction and representation capability. Since contextual information is very critical for deraining, MARB first uses different convolution kernels along with fusion to extract multi-scale rain features and employs feature attention module to identify rain streaks regions and color channels, as well as has the skip connections to aggregate features at multiple levels and accelerate convergence. The proposed method is extensively evaluated on several frequent-use synthetic and real-world datasets. The quantitative and qualitative results show that the designed framework performs better than the recent state-of-the-art deraining approaches on promoting the rain removal performance and preserving image details under various rain streaks cases.
Xi. Chen and Y. Huang—Authors contributed equally to this work, listed alphabetically.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., Yan, S.: Deep joint rain detection and removal from a single image. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1685–1694 (2017)
Yang, W., Tan, R.T., Feng, J., Guo, Z., Yan, S., Liu, J.: Joint rain detection and removal from a single image with contextualized deep networks. IEEE Trans. Pattern Anal. Mach. Intell. 42, 1377–1393 (2020)
Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1715–1723 (2017)
Ren, S., He, K., Girshick, R., Sun, J.: Faster r-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1137–1149 (2017)
Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR 2015: International Conference on Learning Representations 2015 (2015)
Garg, K., Nayar, S.K.: Vision and rain. Int. J. Comput. Vis. 75, 3–27 (2007). https://doi.org/10.1007/s11263-006-0028-6
Zhang, X., Li, H., Qi, Y., Leow, W., Ng, T.: Rain removal in video by combining temporal and chromatic properties. In: 2006 IEEE International Conference on Multimedia and Expo, pp. 461–464 (2006)
Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 3397–3405 (2015)
Li, Y., Tan, R.T., Guo, X., Lu, J., Brown, M.S.: Rain streak removal using layer priors. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2736–2744 (2016)
Gu, S., Meng, D., Zuo, W., Zhang, L.: Joint convolutional analysis and synthesis sparse representation for single image layer separation. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 1717–1725 (2017)
Li, S., et al.: Single image deraining: a comprehensive benchmark analysis. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3838–3847 (2019)
Xu, J., Zhao, W., Liu, P., Tang, X.: Removing rain and snow in a single image using guided filter. In: 2012 IEEE International Conference on Computer Science and Automation Engineering (CSAE), vol. 2, pp. 304–307 (2012)
Zhang, H., Patel, V.M.: Convolutional sparse and low-rank coding-based rain streak removal. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1259–1267 (2017)
Chen, Y.L., Hsu, C.T.: A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In: 2013 IEEE International Conference on Computer Vision, pp. 1968–1975 (2013)
Yang, W., Tan, R.T., Wang, S., Fang, Y., Liu, J.: Single image deraining: from model-based to data-driven and beyond. IEEE Trans. Pattern Anal. Mach. Intell. 1 (2020)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, vol. 27, pp. 2672–2680 (2014)
Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, J.: Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26, 2944–2956 (2017)
Ren, D., Zuo, W., Hu, Q., Zhu, P., Meng, D.: Progressive image deraining networks: a better and simpler baseline. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3937–3946 (2019)
Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 262–277 (2018)
Yang, W., Liu, J., Yang, S., Guo, Z.: Scale-free single image deraining via visibility-enhanced recurrent wavelet learning. IEEE Trans. Image Process. 28, 2948–2961 (2019)
Shen, Y., et al.: Mba-raingan: multi-branch attention generative adversarial network for mixture of rain removal from single images. arXiv preprint arXiv:2005.10582 (2020)
Yan, Y., Ren, W., Guo, Y., Wang, R., Cao, X.: Image deblurring via extreme channels prior. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6978–6986 (2017)
Fu, Y.H., Kang, L.W., Lin, C.W., Hsu, C.T.: Single-frame-based rain removal via image decomposition. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1453–1456 (2011)
Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. IEEE Trans. Circ. Syst. Video Technol. 1 (2019)
Qian, R., Tan, R.T., Yang, W., Su, J., Liu, J.: Attentive generative adversarial network for raindrop removal from a single image. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2482–2491 (2018)
Hu, X., Fu, C.W., Zhu, L., Heng, P.A.: Depth-attentional features for single-image rain removal. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8022–8031 (2019)
Zhang, H., Patel, V.M.: Density-aware single image de-raining using a multi-stream dense network. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 695–704 (2018)
Chen, D., et al.: Gated context aggregation network for image dehazing and deraining. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1375–1383 (2019)
Fu, X., Liang, B., Huang, Y., Ding, X., Paisley, J.: Lightweight pyramid networks for image deraining. IEEE Trans. Neural Netw. 31, 1794–1807 (2020)
Wang, T., Yang, X., Xu, K., Chen, S., Zhang, Q., Lau, R.W.: Spatial attentive single-image deraining with a high quality real rain dataset. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12270–12279 (2019)
Liu, J., Gong, X.: Attention mechanism enhanced LSTM with residual architecture and its application for protein-protein interaction residue pairs prediction. BMC Bioinform. 20, 1–11 (2019)
xu qin, zhilin wang, Bai, Y., Xie, X., Jia, H.: Ffa-net: feature fusion attention network for single image dehazing. In: AAAI 2020 : The Thirty-Fourth AAAI Conference on Artificial Intelligence, vol. 34, pp. 11908–11915 (2020)
Du, Y., Xu, J., Qiu, Q., Zhen, X., Zhang, L.: Variational image deraining. In: 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 2406–2415 (2020)
Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)
Li, S., Ren, W., Zhang, J., Yu, J., Guo, X.: Single image rain removal via a deep decomposition-composition network. Comput. Vis. Image Underst. 186, 48–57 (2019)
Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: ICLR 2015: International Conference on Learning Representations 2015 (2015)
Wang, H., Li, M., Wu, Y., Zhao, Q., Meng, D.: A survey on rain removal from video and single image. arXiv preprint arXiv:1909.08326 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, X., Huang, Y., Xu, L. (2021). Multi-scale Attentive Residual Dense Network for Single Image Rain Removal. In: Ishikawa, H., Liu, CL., Pajdla, T., Shi, J. (eds) Computer Vision – ACCV 2020. ACCV 2020. Lecture Notes in Computer Science(), vol 12623. Springer, Cham. https://doi.org/10.1007/978-3-030-69532-3_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-69532-3_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-69531-6
Online ISBN: 978-3-030-69532-3
eBook Packages: Computer ScienceComputer Science (R0)