Abstract
Video desnowing/deraining plays a vital role in outdoor vision systems, such as autonomous driving and surveillance systems, since the weather conditions significantly degrade their performance. Although numerous approaches have been reported for video snow/rain removal, they are limited to a few videos and did not consider the variations that occurred for the camera and background in real applications. We build a complete snow and rain dataset to overcome this limitation, consisting of 577 videos with synthetic snow and rain, quasi-snow, and real snow and rain. All possible variations of the background and the camera are considered in the dataset. Then, an efficient pixel-wise video desnowing/deraining method is proposed based on the color and temporal information in consecutive video frames. It is highly likely for a single pixel to be a background pixel rather than a snowy pixel at least once in the consecutive frames. Inspiring from this fact along with the color information of the snow pixels, we extract the background pixels from different consecutive frames by searching for the minimum gray-scale intensity. Experimental results demonstrate and validate the proposed method’s robustness to illumination and high-performance static background and camera.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The dataset is available at https://bit.ly/3BHKeRo. For any issues regarding downloading the dataset, please contact arezoo.sadeghzadeh@bahcesehir.edu.tr or bislam.eng@gmail.com.
- 2.
https://www.youtube.com/watch?{v=kNTYEKjXqzs,v=HbgoKKj7TNA}.
References
Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, J.: Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26(6), 2944–2956 (2017)
Garg, K., Nayar, S.K.: Detection and removal of rain from videos. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004, vol. 1, pp. I–I. IEEE (2004)
Garg, K., Nayar, S.K.: When does a camera see rain? In: Tenth IEEE International Conference on Computer Vision (ICCV 2005) Volume 1, vol. 2, pp. 1067–1074. IEEE (2005)
Garg, K., Nayar, S.K.: Vision and rain. Int. J. Comput. Vis. 75(1), 3–27 (2007)
Jaw, D.W., Huang, S.C., Kuo, S.Y.: DesnowGAN: an efficient single image snow removal framework using cross-resolution lateral connection and GANs. IEEE Trans. Circuits Syst. Video Technol. 31(4), 1342–1350 (2020)
Jeny, A.A., Sakib, A.N.M., Junayed, M.S., Lima, K.A., Ahmed, I., Islam, M.B.: SkNet: a convolutional neural networks based classification approach for skin cancer classes. In: 2020 23rd International Conference on Computer and Information Technology (ICCIT), pp. 1–6. IEEE (2020)
Jiang, T.X., Huang, T.Z., Zhao, X.L., Deng, L.J., Wang, Y.: A novel tensor-based video rain streaks removal approach via utilizing discriminatively intrinsic priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4057–4066 (2017)
Jung, A.B., et al.: imgaug (2020). https://github.com/aleju/imgaug. Accessed 01 Feb 2020
Kim, J.H., Sim, J.Y., Kim, C.S.: Stereo video deraining and desnowing based on spatiotemporal frame warping. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 5432–5436. IEEE (2014)
Li, M., Cao, X., Zhao, Q., Zhang, L., Gao, C., Meng, D.: Video rain/snow removal by transformed online multiscale convolutional sparse coding. arXiv preprint arXiv:1909.06148 (2019)
Li, M., Cao, X., Zhao, Q., Zhang, L., Meng, D.: Online rain/snow removal from surveillance videos. IEEE Trans. Image Process. 30, 2029–2044 (2021)
Li, M., et al.: Video rain streak removal by multiscale convolutional sparse coding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6644–6653 (2018)
Li, R., Cheong, L.F., Tan, R.T.: Heavy rain image restoration: Integrating physics model and conditional adversarial learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1633–1642 (2019)
Liu, J., Yang, W., Yang, S., Guo, Z.: Erase or fill? Deep joint recurrent rain removal and reconstruction in videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3233–3242 (2018)
Liu, Y.F., Jaw, D.W., Huang, S.C., Hwang, J.N.: DesnowNet: context-aware deep network for snow removal. IEEE Trans. Image Process. 27(6), 3064–3073 (2018)
Pitropov, M., et al.: Canadian adverse driving conditions dataset. Int. J. Robot. Res. 40(4–5), 681–690 (2021)
Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018)
Ren, W., Tian, J., Han, Z., Chan, A., Tang, Y.: Video desnowing and deraining based on matrix decomposition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4210–4219 (2017)
Wei, W., Yi, L., Xie, Q., Zhao, Q., Meng, D., Xu, Z.: Should we encode rain streaks in video as deterministic or stochastic? In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2516–2525 (2017)
Yang, W., Liu, J., Feng, J.: Frame-consistent recurrent video deraining with dual-level flow. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1661–1670 (2019)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Sadeghzadeh, A., Islam, M.B., Zaker, R. (2021). An Efficient Video Desnowing and Deraining Method with a Novel Variant Dataset. In: Vincze, M., Patten, T., Christensen, H.I., Nalpantidis, L., Liu, M. (eds) Computer Vision Systems. ICVS 2021. Lecture Notes in Computer Science(), vol 12899. Springer, Cham. https://doi.org/10.1007/978-3-030-87156-7_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-87156-7_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87155-0
Online ISBN: 978-3-030-87156-7
eBook Packages: Computer ScienceComputer Science (R0)