Abstract
In recent years, the explosive growth of video data has posed a challenge to the performance of traditional video coding frameworks. The industry is thus faced with the pressing issue of how to transmit high-quality videos under limited bandwidth conditions. To address this challenge, a super-resolution reconstruction video coding scheme has been proposed, which combines traditional coding frameworks with deep learning-based super-resolution technology. By reducing the data coding volume through pre-coding downsampling and reconstructing videos through post-decoding upsampling, this scheme shows great potential in solving the aforementioned problems. However, previous super-resolution reconstruction video coding schemes have failed to effectively utilize the inter-frame correlation of video sequences, which limits the coding efficiency of the scheme. To overcome this limitation, this paper proposes an upsampling reconstruction network based on inter-frame information exploration. Experimental results show that, compared with the HEVC standard, the proposed scheme achieves a reduction of 11.05%, 11.3%, and 8.84% in the BD-BR index under the All-Intra, Low Delay P, and Random Access coding configurations, respectively, demonstrating higher coding efficiency than the previous human-designed scheme.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and materials
The data that support the findings of this study are available from the corresponding author upon reasonable request.
References
Liou M (1991) Overview of the p\(\times \) 64 kbit/s video coding standard. Commun ACM 34(4):59–63
Shlien S (1994) Guide to mpeg-1 audio standard. IEEE Trans Broadcast 40(4):206–218
Rijkse KH (1996) 263: Video coding for low-bit-rate communication. IEEE Commun Mag 34(12):42–45
Wiegand T, Sullivan GJ, Bjontegaard G, Luthra A (2003) Overview of the h. 264/avc video coding standard. IEEE Trans Circ Syst Vid Technol 13(7):560–576
Sullivan GJ, Ohm J-R, Han W-J, Wiegand T (2012) Overview of the high efficiency video coding (hevc) standard. IEEE Trans Circ Syst Vid Technol 22(12):1649–1668
Bross B, Wang Y-K, Ye Y, Liu S, Chen J, Sullivan GJ, Ohm J-R (2021) Overview of the versatile video coding (vvc) standard and its applications. IEEE Trans Circ Syst Vid Technol 31(10):3736–3764
Cisco U (2021) Cisco annual internet report (2018–2023) white paper. 2020. Acessado em 10(01), pp 1–35
Shen M, Xue P, Wang C (2011) Down-sampling based video coding using super-resolution technique. IEEE Trans Circ uits and Syst Vid Technol 21(6):755–765
Ho M-M, He G, Wang Z, Zhou J (2020) Down-sampling based video coding with degradation-aware restoration-reconstruction deep neural network. In: Multimedia modeling: 26th International conference, MMM 2020, Daejeon, South Korea, January 5–8, 2020, Proceedings, Part I 26, pp 99–110. Springer
Ho MM, Zhou J, He G (2021) Rr-dncnn v2. 0: enhanced restoration-reconstruction deep neural network for down-sampling-based video coding. IEEE Trans Image Process 30:1702–1715
Chan KC, Wang X, Yu K, Dong C, Loy CC (2021) Basicvsr: The search for essential components in video super-resolution and beyond. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 4947–4956
Tian Y, Zhang Y, Fu Y, Xu C (2020) Tdan: Temporally-deformable alignment network for video super-resolution. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 3360–3369
Huang Y, Wang W, Wang L (2017) Video super-resolution via bidirectional recurrent convolutional networks. IEEE Trans Pattern Anal Mach Intell 40(4):1015–1028
Wang X, Chan KC, Yu K, Dong C, Change Loy, C (2019) Edvr: Video restoration with enhanced deformable convolutional networks. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition workshops, pp 0–0
Chen Y, Murherjee D, Han J, Grange A, Xu Y, Liu Z, Parker S, Chen C, Su H, Joshi U, et al (2018) An overview of core coding tools in the av1 video codec. In: 2018 Picture Coding Symposium (PCS), pp 41–45. IEEE
Lin C, Li Y, Zhang K, Zhang Z, Zhang L (2021) Cnn-based super resolution for video coding using decoded information. In: 2021 International Conference on Visual Communications and Image Processing (VCIP), pp 1–5. IEEE
Montgomery C, Lars H (1994) Xiph. org video test media (derf’s collection). the xiph open source community
Bjontegaard G (2001) Calculation of average psnr differences between rd-curves. ITU SG16 Doc. VCEG-M33
Acknowledgements
The work is supported by the National Key Research and Development Program of China(2022YFF0607000), National Natural Science Foundation of China (61871188), Guangdong Basic and Applied Basic Research Foundation (2023A1515010993), Guangdong Provincial Key Laboratory of Human Digital Twin (2022B1212010004), Guangzhou City Science and Technology Research Projects (2023B01J0011).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
We declared there is no conflict of interest in terms of this submission.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Cao, Y., Xu, H., Zhou, Z. et al. Super-resolution reconstructed video coding scheme based on inter-frame information. Multimed Tools Appl 83, 47847–47863 (2024). https://doi.org/10.1007/s11042-023-17441-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-023-17441-z