Skip to main content
Springer Nature Link
Log in

DSCVSR: A Lightweight Video Super-Resolution for Arbitrary Magnification

  • Conference paper
  • First Online:
Knowledge Science, Engineering and Management (KSEM 2024)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14884))

  • 604 Accesses

Abstract

Video super-resolution, a fundamental task in the field of computer vision, has gained much attention and performance in recent years. However, since deep learning introduces a large number of parameters, which can result in a large resource overhead, the model cannot be deployed on edge devices. Therefore, in this paper, we design a lightweight video super-resolution model, named Depthwise Separable Convolutional Video Super-Resolution (DSCVSR), which utilizes a continuous memory mechanism by constructing a dense depthwise separable convolutional residual block to fuse the deep and shallow feature information in order to enable the network to better learn the details in the video, and also constructs an information-filling module to solve the problem of information loss brought about by the depthwise separable convolution, as well as designing an information-filling module to solve the problem of information loss brought about by the depthwise separable convolution information loss problem caused by deep separable convolution, and a knowledge distillation loss is designed to migrate the knowledge from the teacher’s model to the model to achieve the superscoring results with arbitrary multiplicity. In the experiments, the method is tested on common video datasets, and it is verified that the proposed method can achieve good results with a small number of parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Caballero, J., et al.: Real-time video super-resolution with spatio-temporal networks and motion compensation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4778–4787 (2017)

    Google Scholar 

  2. Cao, W., Li, D., Zhang, X., Qiu, M., Liu, Y.: BLSHF: broad learning system with hybrid features. In: Memmi, G., Yang, B., Kong, L., Zhang, T., Qiu, M. (eds.) KSEM 2022. LNCS, vol. 13369, pp. 655–666. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-10986-7_53

    Chapter  Google Scholar 

  3. Cao, W., et al.: A review on multimodal zero-shot learning. Wiley Interdisc. Rev. Data Mining Knowl. Discov. 13(2), e1488 (2023)

    Article  Google Scholar 

  4. Chan, K.C., Wang, X., Yu, K., Dong, C., Loy, C.C.: Basicvsr: the search for essential components in video super-resolution and beyond. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4947–4956 (2021)

    Google Scholar 

  5. Chan, K.C., Zhou, S., Xu, X., Loy, C.C.: Basicvsr++: improving video super-resolution with enhanced propagation and alignment. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5972–5981 (2022)

    Google Scholar 

  6. Chan, K.C., Zhou, S., Xu, X., Loy, C.C.: Investigating tradeoffs in real-world video super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5962–5971 (2022)

    Google Scholar 

  7. Chen, P., Liu, S., Zhao, H., Jia, J.: Distilling knowledge via knowledge review. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5008–5017 (2021)

    Google Scholar 

  8. Chen, Z., et al.: Videoinr: learning video implicit neural representation for continuous space-time super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2047–2057 (2022)

    Google Scholar 

  9. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2015)

    Article  Google Scholar 

  10. Guo, J., et al.: Distilling object detectors via decoupled features. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2154–2164 (2021)

    Google Scholar 

  11. Haris, M., Shakhnarovich, G., Ukita, N.: Space-time-aware multi-resolution video enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2859–2868 (2020)

    Google Scholar 

  12. Hong, Z., et al.: MetaVSR: a novel approach to video super-resolution for arbitrary magnification. In: Rudinac, S., et al. (eds.) MMM 2024. LNCS, vol. 14554, pp. 300–313. Springer, Cham (2024). https://doi.org/10.1007/978-3-031-53305-1_23

    Chapter  Google Scholar 

  13. Howard, A.G., et al.: Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

  14. Huang, Y., Chen, J.: Improved EDVR model for robust and efficient video super-resolution. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 103–111 (2022)

    Google Scholar 

  15. Isobe, T., Jia, X., Gu, S., Li, S., Wang, S., Tian, Q.: Video super-resolution with recurrent structure-detail network. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12357, pp. 645–660. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58610-2_38

    Chapter  Google Scholar 

  16. Isobe, T., et al.: Video super-resolution with temporal group attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8008–8017 (2020)

    Google Scholar 

  17. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43

    Chapter  Google Scholar 

  18. Kappeler, A., Yoo, S., Dai, Q., Katsaggelos, A.K.: Video super-resolution with convolutional neural networks. IEEE Trans. Comput. Imaging 2(2), 109–122 (2016)

    Article  MathSciNet  Google Scholar 

  19. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  20. Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 624–632 (2017)

    Google Scholar 

  21. Li, G., Li, X., Wang, Y., Zhang, S., Wu, Y., Liang, D.: Knowledge distillation for object detection via rank mimicking and prediction-guided feature imitation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 1306–1313 (2022)

    Google Scholar 

  22. Liu, C., Sun, D.: On Bayesian adaptive video super resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 346–360 (2013)

    Article  Google Scholar 

  23. Miles, R., Yucel, M.K., Manganelli, B., Saà-Garriga, A.: Mobilevos: real-time video object segmentation contrastive learning meets knowledge distillation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10480–10490 (2023)

    Google Scholar 

  24. Nah, S., et al.: NTIRE 2019 challenge on video deblurring and super-resolution: dataset and study. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (2019)

    Google Scholar 

  25. Polino, A., Pascanu, R., Alistarh, D.: Model compression via distillation and quantization. arXiv preprint arXiv:1802.05668 (2018)

  26. Tao, X., Gao, H., Liao, R., Wang, J., Jia, J.: Detail-revealing deep video super-resolution. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4472–4480 (2017)

    Google Scholar 

  27. Wang, H., Su, D., Liu, C., Jin, L., Sun, X., Peng, X.: Deformable non-local network for video super-resolution. IEEE Access 7, 177734–177744 (2019)

    Article  Google Scholar 

  28. Wang, X., Chan, K.C., Yu, K., Dong, C., Change Loy, C.: EDVR: video restoration with enhanced deformable convolutional networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (2019)

    Google Scholar 

  29. Xue, T., Chen, B., Wu, J., Wei, D., Freeman, W.T.: Video enhancement with task-oriented flow. Int. J. Comput. Vision 127, 1106–1125 (2019)

    Article  Google Scholar 

  30. Zhang, L., Ma, K.: Improve object detection with feature-based knowledge distillation: towards accurate and efficient detectors. In: International Conference on Learning Representations (2020)

    Google Scholar 

  31. Zhou, X., Cao, W., Gao, H., Ming, Z., Zhang, J.: STI-Net: spatiotemporal integration network for video saliency detection. Inf. Sci. 628, 134–147 (2023)

    Article  Google Scholar 

Download references

Acknowledgement

This work was supported by the National Natural Science Foundation of China (62106150), the Open Research Fund of Anhui Province Key Laboratory of Machine Vision Inspection (KLMVI-2023-HIT-01), and the Director Fund of Guangdong Laboratory of Artificial Intelligence and Digital Economy (Shenzhen) (24420001).

Author information

Authors and Affiliations

  1. College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China

    Zixuan Hong, Zhiwu Xu & Zhong Ming

  2. Guangdong Laboratory of Artificial Intelligence and Digital Economy (Shenzhen), Shenzhen, 518107, China

    Zixuan Hong, Weipeng Cao & Zhong Ming

  3. Anhui Province Key Laboratory of Machine Vision Inspection, Yangtze River Delta HIT Robot Technology Research Institute, Wuhu, 241000, China

    Weipeng Cao, Chuqing Cao & Liang Zheng

Authors
  1. Zixuan Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weipeng Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiwu Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhong Ming
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chuqing Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Liang Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weipeng Cao .

Editor information

Editors and Affiliations

  1. Chinese Academy of Sciences, Beijing, China

    Cungeng Cao

  2. Zhejiang University, Zhejiang, China

    Huajun Chen

  3. Emory University, Atlanta, GA, USA

    Liang Zhao

  4. Birmingham City University, Birmingham, UK

    Junaid Arshad

  5. Monash University, Banten, Indonesia

    Taufiq Asyhari

  6. Birmingham City University, Birmingham, UK

    Yonghao Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hong, Z., Cao, W., Xu, Z., Ming, Z., Cao, C., Zheng, L. (2024). DSCVSR: A Lightweight Video Super-Resolution for Arbitrary Magnification. In: Cao, C., Chen, H., Zhao, L., Arshad, J., Asyhari, T., Wang, Y. (eds) Knowledge Science, Engineering and Management. KSEM 2024. Lecture Notes in Computer Science(), vol 14884. Springer, Singapore. https://doi.org/10.1007/978-981-97-5492-2_9

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-5492-2_9

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-5491-5

  • Online ISBN: 978-981-97-5492-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics