Skip to main content
Springer Nature Link
Log in

A Novel Encoder-Decoder Network with Multi-domain Information Fusion for Video Deblurring

  • Conference paper
  • First Online:
Pattern Recognition (ICPR 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15332))

Included in the following conference series:

  • 180 Accesses

Abstract

Due to various challenging conditions during video recording, such as camera shake and out-of-focus issues, video deblurring remains a difficult problem. To address this, we propose the Spatial-Temporal Frequency domain Fusion network (STFFNet) and improve the network from three key aspects. Firstly, we introduce the Encoder-Decoder idea to create a novel backbone to combine global and local features effectively. Secondly, a new feature fusion module that focuses on the differences between frames is proposed to help better deblur the current frame. Finally, STFFNet introduces a Frequency Domain Converter (FDC) to transform the image information from the spatial domain to the frequency domain, enhancing image restoration by narrowing the gap between the deblurring and ground truth images in the frequency domain. Experimental results demonstrate that the proposed method achieves state-of-the-art deblurring performance on benchmark datasets. The code is available at: https://github.com/Paige-Norton/STFFNet.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Chan, K.C., Zhou, S., Xu, X., Loy, C.C.: BasicVSR++: improving video super-resolution with enhanced propagation and alignment. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5972–5981 (2022)

    Google Scholar 

  2. Haris, M., Shakhnarovich, G., Ukita, N.: Space-time-aware multi-resolution video enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2859–2868 (2020)

    Google Scholar 

  3. Liu, X., Kong, L., Zhou, Y., Zhao, J., Chen, J.: End-to-end trainable video super-resolution based on a new mechanism for implicit motion estimation and compensation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2416–2425 (2020)

    Google Scholar 

  4. Shi, Z., Liu, X., Shi, K., Dai, L., Chen, J.: Video frame interpolation via generalized deformable convolution. IEEE Trans. Multimedia 24, 426–439 (2021)

    Article  Google Scholar 

  5. Shi, Z., Xu, X., Liu, X., Chen, J., Yang, M.H.: Video frame interpolation transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17482–17491 (2022)

    Google Scholar 

  6. Zhang, F., Li, Y., You, S., Fu, Y.: Learning temporal consistency for low light video enhancement from single images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4967–4976 (2021)

    Google Scholar 

  7. Su, S., Delbracio, M., Wang, J., Sapiro, G., Heidrich, W., Wang, O.: Deep video deblurring for hand-held cameras. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1279–1288 (2017)

    Google Scholar 

  8. Wang, X., Chan, K.C., Yu, K., Dong, C., Change Loy, C.: EDVR: video restoration with enhanced deformable convolutional networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (2019)

    Google Scholar 

  9. Nah, S., Son, S., Lee, K.M.: Recurrent neural networks with intra-frame iterations for video deblurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8102–8111 (2019)

    Google Scholar 

  10. Zhou, S., Zhang, J., Pan, J., Xie, H., Zuo, W., Ren, J.: Spatio-temporal filter adaptive network for video deblurring. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2482–2491 (2019)

    Google Scholar 

  11. Hyun Kim, T., Mu Lee, K., Scholkopf, B., Hirsch, M.: Online video deblurring via dynamic temporal blending network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4038–4047 (2017)

    Google Scholar 

  12. Wieschollek, P., Hirsch, M., Scholkopf, B., Lensch, H.: Learning blind motion deblurring. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 231–240 (2017)

    Google Scholar 

  13. Du, B., Ren, X., Ren, J.: CNN-based image super-resolution and deblurring. In: Proceedings of the 2019 International Conference on Video, Signal and Image Processing, pp. 70–74 (2019)

    Google Scholar 

  14. Cai, J., Zuo, W., Zhang, L.: Dark and bright channel prior embedded network for dynamic scene deblurring. IEEE Trans. Image Process. 29, 6885–6897 (2020)

    Article  Google Scholar 

  15. Zhang, X., Wang, T., Jiang, R., Zhao, L., Xu, Y.: Multi-attention convolutional neural network for video deblurring. IEEE Trans. Circuits Syst. Video Technol. 32(4), 1986–1997 (2021)

    Article  Google Scholar 

  16. Ren, W., et al.: Deblurring dynamic scenes via spatially varying recurrent neural networks. IEEE Trans. Pattern Anal. Mach. Intell. 44(8), 3974–3987 (2021)

    Google Scholar 

  17. Zhu, C., et al.: Deep recurrent neural network with multi-scale bi-directional propagation for video deblurring. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 3598–3607 (2022)

    Google Scholar 

  18. Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015)

  19. Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)

  20. Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, vol. 27 (2014)

    Google Scholar 

  21. Cai, M., Zhang, H., Huang, H., Geng, Q., Li, Y., Huang, G.: Frequency domain image translation: more photo-realistic, better identity-preserving. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13930–13940 (2021)

    Google Scholar 

  22. Jiang, J., et al.: Five a+ network: you only need 9k parameters for underwater image enhancement. arXiv preprint arXiv:2305.08824 (2023)

  23. Cooley, J.W., Tukey, J.W.: An algorithm for the machine calculation of complex fourier series. Math. Comput. 19(90), 297–301 (1965)

    Article  MathSciNet  Google Scholar 

  24. Zhong, Z., Gao, Y., Zheng, Y., Zheng, B., Sato, I.: Real-world video deblurring: a benchmark dataset and an efficient recurrent neural network. Int. J. Comput. Vision 131(1), 284–301 (2023)

    Article  Google Scholar 

  25. Nah, S., Hyun Kim, T., Mu Lee, K.: Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3883–3891 (2017)

    Google Scholar 

  26. Nah, S., et al.: NTIRE 2019 challenge on video deblurring and super-resolution: Dataset and study. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (2019)

    Google Scholar 

  27. Tao, X., Gao, H., Shen, X., Wang, J., Jia, J.: Scale-recurrent network for deep image deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8174–8182 (2018)

    Google Scholar 

  28. Pan, J., Bai, H., Tang, J.: Cascaded deep video deblurring using temporal sharpness prior. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3043–3051 (2020)

    Google Scholar 

  29. Li, D., Zhang, Y., Cheung, K.C., Wang, X., Qin, H., Li, H.: Learning degradation representations for image deblurring. In: European Conference on Computer Vision, pp. 736–753. Springer (2022)

    Google Scholar 

  30. Mao, X., Liu, Y., Liu, F., Li, Q., Shen, W., Wang, Y.: Intriguing findings of frequency selection for image deblurring. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 1905–1913 (2023)

    Google Scholar 

Download references

Acknowledgements

This work is partly supported by the Chongqing University of Technology high-quality development Action Plan for of graduate education (gzlcx20233188), the Chongqing Postgraduate Research and Innovation Project Funding (No. CYS23677) and the Youth Project of Science and Technology Research Program of Chongqing Education Commission of China (No. KJQN202401106).

Author information

Authors and Affiliations

  1. Chongqing University of Technology, Chongqing, 400054, China

    Peiqi Xie, Jinhong He, Chengyun Song & Minglong Xue

Authors
  1. Peiqi Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinhong He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chengyun Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Minglong Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Minglong Xue .

Editor information

Editors and Affiliations

  1. University of Salford, Salford, Lancashire, UK

    Apostolos Antonacopoulos

  2. Indian Institute of Technology Bombay, Mumbai, Maharashtra, India

    Subhasis Chaudhuri

  3. Johns Hopkins University, Baltimore, MD, USA

    Rama Chellappa

  4. Chinese Academy of Sciences, Beijing, China

    Cheng-Lin Liu

  5. Indian Institute of Technology Kharagpur, Kharagpur, West Bengal, India

    Saumik Bhattacharya

  6. Indian Statistical Institute, Kolkata, West Bengal, India

    Umapada Pal

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xie, P., He, J., Song, C., Xue, M. (2025). A Novel Encoder-Decoder Network with Multi-domain Information Fusion for Video Deblurring. In: Antonacopoulos, A., Chaudhuri, S., Chellappa, R., Liu, CL., Bhattacharya, S., Pal, U. (eds) Pattern Recognition. ICPR 2024. Lecture Notes in Computer Science, vol 15332. Springer, Cham. https://doi.org/10.1007/978-3-031-78125-4_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-78125-4_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-78124-7

  • Online ISBN: 978-3-031-78125-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships