Abstract
A deep learning-based single image deraining algorithm is proposed in this work. Instead of modeling a rain layer as a linear function between the rain image and its clear version as previous works do, we directly formulate the clear image as the result of a non-linear mapping of thrain image. We construct a coarse deraining convolutional network and a refinement convolutional network to learn this non-linear mapping function. The coarse deraining network is trained to detect the rain streaks with different directions, and restore a raw derained result. The refinement network aims at refining the result according to the raw derained image and the original rain image. By combining the two networks, we are able to well-restore the rain-free image. Experimental results demonstrate that the proposed deraining method can produce high-quality clear images from both synthetic and real-world rain images, outperforming the state-of-the-art methods qualitatively and quantitatively.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adding rain to a photo with Photoshop. http://www.photoshopessentials.com/photo-effects/rain/
Bossu J, Hautiere N, Tarel JP (2011) Rain or snow detection in image sequences through use of a histogram of orientation of streaks. Int J Comput Vis 93(3):348–367
Cai B, Xu X, Jia K, Qing C, Tao D (2016) DehazeNet: an end-to-end system for single image haze removal. IEEE Trans Image Process 25(11):5187–5198
Chen LC, Barron JT, Papandreou G, Murphy K, Yuille AL (2016) Semantic image segmentation with task-specific edge detection using CNNs and a discriminatively trained domain transform. In: IEEE conference on computer vision and pattern recognition
Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307
Garg K, Nayar SK (2004) Detection and removal of rain from videos. In: IEEE conference on computer vision and pattern recognition
Garg K, Nayar SK (2007) Vision and rain. Int J Comput Vis 75(1):3–27
Garg K, Nayar SK (2005) When does a camera see rain?. In: IEEE conference on computer vision
He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. In: European conference on computer vision, pp 346–361
Hong S, You T, Han B (2015) Online tracking by learning discriminative saliency map with convolutional neural network. In: International conference on machine learning
Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: Convolutional architecture for fast feature embedding. In: ACM international conference on multimedia, pp 675–678
Kang LW, Lin CW, Fu YH (2012) Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans Image Process 21(4):1742–1755
Kim JH, Lee C, Sim JY, Kim CS (2013) Single image deraining using an adaptive nonlocal means filter. In: IEEE international conference on image processing
Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
Li Y, Tan RT, Guo X, Lu J, Brown MS (2016) Rain streak removal using layer priors. In: IEEE conference computer vision and pattern recognition
Luo Y, Xu Y, Ji H (2015) Removing rain from a single image via discriminative sparse coding. In: IEEE international conference on computer vision
Ren W, Liu S, Zhang H, Pan J, Cao X, Yang MH (2004) Single image dehazing via multi-scale convolutional neural networks. In: IEEE conference on computer vision and pattern recognition
Sheikh HR, Bovik AC (2006) Image information and visual quality. IEEE Trans Image Process 15(2):430–444
Thu QH, Ghanbari M (2008) Scope of validity of psnr in image/video quality assessment. Electron Lett 44(13):800–801
Tripathi AK, Mukhopadhyay S (2014) Removal of rain from videos: a review. Signal, Image and Video Processing, Springer 8(8):1421–1430
Vincent P, Larochelle H, Lajoie I, Bengio Y, Manazagol PA (2010) Stacked denoising autoencoders: learning useful representations in a deep neural network with a local denoising criterion. J Mach Learn Res 11(12):3371–3408
Wang Z, Bovik AC (2002) A universal image quality index. IEEE Trans Image Signal Process Lett 19(3):81–84
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13 (4):600–612
Zhang X, Li H, Qi Y, Leow WK, Ng TK (2006) Rain removal in video by combining temporal and chromatic properties. In: IEEE international conference on multimedia and expo
Acknowledgments
This work is financially supported by National Natural Science Foundation of China (61202269, 61472089, 61202293, 31600591), Science and Technology Plan Project of Guangdong Province (2014A050503057, 2015A020209124, 2016A020210087).
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Wang, M., Mai, J., Cai, R. et al. Single image deraining using deep convolutional networks. Multimed Tools Appl 77, 25905–25918 (2018). https://doi.org/10.1007/s11042-018-5825-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-018-5825-8