Abstract
Snowfall severely degrades outdoor video visibility while reducing the performance of subsequent vision tasks. Although video recovery methods based on deep learning have achieved amazing accomplishments, video snow removal still faces problems such as varying scales and intricate trajectories of snowflakes, which makes it difficult to remove snowflakes and easy to create artifacts on moving objects. To address these issues, we propose a deformable multi-scale video desnowing network. Specifically, we design a multi-scale pseudo-3D residual block(MSRB-P3D) that can effectively remove snowflakes of different scales. Furthermore, a deformable large kernel attention 3Dblock(D-LKA 3Dblock) is introduced to capture the inter-frame dynamic information and reduce the artifacts. Due to the scarcity of dataset, we proposed a new dataset named Synthetic and Real Snowy Video Dataset(SRSVD). Extensive experiments have proven that our proposed method not only outperforms other state-of-the-art methods on both synthetic and real snowy videos, but also effectively improves performance on subsequent vision task.
G. Zhou—This work was supported by National Natural Science Foundation of China (No.62166040, No.62261053) and Tianshan Talent Training Project - Xinjiang Science and Technology Innovation Team Program (2023TSYCTD0012).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bossu, J., Hautiere, N., Tarel, J.P.: Rain or snow detection in image sequences through use of a histogram of orientation of streaks. Int. J. Comput. Vision 93, 348–367 (2011)
Kim, J.H., Sim, J.Y., Kim, C.S.: Video deraining and desnowing using temporal correlation and low-rank matrix completion. IEEE Trans. Image Process. 24(9), 2658–2670 (2015)
Li, M., et al.: Video rain streak removal by multiscale convolutional sparse coding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6644–6653 (2018)
Li, M., Cao, X., Zhao, Q., Zhang, L., Meng, D.: Online rain/snow removal from surveillance videos. IEEE Trans. Image Process. 30, 2029–2044 (2021)
Ren, W., Tian, J., Han, Z., Chan, A., Tang, Y.: Video desnowing and deraining based on matrix decomposition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4210–4219 (2017)
Yue, Z., Xie, J., Zhao, Q., Meng, D.: Semi-supervised video deraining with dynamical rain generator. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 642–652 (2021)
Zhang, K., Li, D., Luo, W., Ren, W., Liu, W.: Enhanced spatio-temporal interaction learning for video deraining: faster and better. IEEE Trans. Pattern Anal. Mach. Intell. 45(1), 1287–1293 (2022)
Chen, H., et al.: Snow removal in video: a new dataset and a novel method. In: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 13165–13176. IEEE (2023)
Liu, Y.F., Jaw, D.W., Huang, S.C., Hwang, J.N.: DesnowNet: context-aware deep network for snow removal. IEEE Trans. Image Process. 27(6), 3064–3073 (2018)
Chen, W.T., et al.: All snow removed: single image desnowing algorithm using hierarchical dual-tree complex wavelet representation and contradict channel loss. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4196–4205 (2021)
Zhang, K., Li, R., Yu, Y., Luo, W., Li, C.: Deep dense multi-scale network for snow removal using semantic and depth priors. IEEE Trans. Image Process. 30, 7419–7431 (2021)
Chen, X., et al.: Unpaired deep image deraining using dual contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2017–2026 (2022)
Chen, W.-T., Fang, H.-Y., Ding, J.-J., Tsai, C.-C., Kuo, S.-Y.: JSTASR: joint size and transparency-aware snow removal algorithm based on modified partial convolution and veiling effect removal. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12366, pp. 754–770. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58589-1_45
Yang, W., Tan, R.T., Wang, S., Liu, J.: Self-learning video rain streak removal: when cyclic consistency meets temporal correspondence. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1720–1729 (2020)
Wang, S., Zhu, L., Fu, H., Qin, J., Schönlieb, C.B., Feng, W., Wang, S.: Rethinking video rain streak removal: a new synthesis model and a deraining network with video rain prior. In: European Conference on Computer Vision, pp. 565–582. Springer (2022)
Yang, Y., Aviles-Rivero, A.I., Fu, H., Liu, Y., Wang, W., Zhu, L.: Video adverse-weather-component suppression network via weather messenger and adversarial backpropagation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13200–13210 (2023)
Xue, T., Zhou, G., He, R., Wang, Z., Chen, J., Jia, Z.: RVDNet: a two-stage network for real-world video desnowing with domain adaptation. In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3305–3309. IEEE (2024)
Christiansen, M.: Adobe after effects CC visual effects and compositing studio techniques. Adobe Press (2013)
Azad, R., et al.: Beyond self-attention: deformable large kernel attention for medical image segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1287–1297 (2024)
Tong, Z., Song, Y., Wang, J., Wang, L.: VideoMAE: masked autoencoders are data-efficient learners for self-supervised video pre-training. Adv. Neural. Inf. Process. Syst. 35, 10078–10093 (2022)
Ying, X., Wang, L., Wang, Y., Sheng, W., An, W., Guo, Y.: Deformable 3d convolution for video super-resolution. IEEE Signal Process. Lett. 27, 1500–1504 (2020)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc.: Ser. B (Methodol.) 39(1), 1–22 (1977)
Uhlenbeck, G.E., Ornstein, L.S.: On the theory of the Brownian motion. Phys. Rev. 36(5), 823 (1930)
Jonathon Luiten, A.H.: Trackeval (2020). https://github.com/JonathonLuiten/TrackEval
Chan, K.C., Zhou, S., Xu, X., Loy, C.C.: BasicVSR++: improving video super-resolution with enhanced propagation and alignment. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5972–5981 (2022)
Chen, S., Ye, T., Liu, Y., Chen, E., Shi, J., Zhou, J.: SnowFormer: Scale-aware transformer via context interaction for single image desnowing. arXiv preprint arXiv:2208.09703 (2022)
Milan, A., Leal-Taixé, L., Reid, I., Roth, S., Schindler, K.: MOT16: A benchmark for multi-object tracking. arXiv:1603.00831 [cs] (2016). http://arxiv.org/abs/1603.00831, arXiv: 1603.00831
Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the clear mot metrics. EURASIP J. Image Video Process. 2008, 1–10 (2008)
Luiten, J., Osep, A., Dendorfer, P., Torr, P., Geiger, A., Leal-Taixé, L., Leibe, B.: HOTA: a higher order metric for evaluating multi-object tracking. In: International Journal of Computer Vision, pp. 1–31 (2020)
Ristani, E., Solera, F., Zou, R., Cucchiara, R., Tomasi, C.: Performance measures and a data set for multi-target, multi-camera tracking. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 17–35. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48881-3_2
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_38
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
He, R., Zhou, G., Xue, T., Liu, Z., Jia, Z. (2025). Deformable Multi-Scale Network for Snow Removal in Video. In: Antonacopoulos, A., Chaudhuri, S., Chellappa, R., Liu, CL., Bhattacharya, S., Pal, U. (eds) Pattern Recognition. ICPR 2024. Lecture Notes in Computer Science, vol 15332. Springer, Cham. https://doi.org/10.1007/978-3-031-78125-4_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-78125-4_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-78124-7
Online ISBN: 978-3-031-78125-4
eBook Packages: Computer ScienceComputer Science (R0)