Abstract
Deinterlacing is a classical issue in video processing, aimed at generating progressive video from interlaced content. There are precious videos that are difficult to reshoot and still contain interlaced content. Previous methods have primarily focused on simple interlaced mechanisms and have struggled to handle the complex artifacts present in real-world early videos. Therefore, we propose a Transformer-based method for deinterlacing, which consists of a Feature Extractor, a De-Transformer, and a Residual DenseNet module. By incorporating self-attention in Transformer, our proposed method is able to better utilize the inter-frame movement correlation. Additionally, we combine a properly designed loss function and residual blocks to train an end-to-end deinterlacing model. Extensive experimental results on various video sequences demonstrate that our proposed method outperforms state-of-the-art methods in different tasks by up to 1.41\(\sim \)2.64dB. Furthermore, we also discuss several related issues, such as the rationality of the network structure. The code for our proposed method is available at https://github.com/Anonymous2022-cv/DeT.git.
The authors are grateful to Zhejiang Gongshang University for their valuable computing resources and outstanding laboratory facilities, as well as the support from the Zhejiang Provincial Natural Science Foundation of China(Grant No. LY22F020013), National Natural Science Foundation of China (Grant No. 62172366), “Pioneer” and “Leading Goose” R & D Program of Zhejiang Province (2023C01150), and “Digital+” Discipline Construction Project of Zhejiang Gongshang University (Grant No. SZJ2022B009).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bernasconi, M., Djelouah, A., Hattori, S., Schroers, C.: Deep deinterlacing. In: SMPTE Annual Technical Conference and Exhibition, pp. 1–12 (2020)
Bhakar, V., Agur, A., Digalwar, A., Sangwan, K.S.: Life cycle assessment of crt, lcd and led monitors. Procedia CIRP 29, 432–437 (2015)
Caballero, J., et al.: Real-time video super-resolution with spatio-temporal networks and motion compensation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4778–4787 (2017)
Chen, M.J., Huang, C.H., Hsu, C.T.: Efficient de-interlacing technique by inter-field information. IEEE Trans. Consum. Electron. 50(4), 1202–1208 (2004)
d’Aspremont, A.: Smooth optimization with approximate gradient. SIAM J. Optim. 19(3), 1171–1183 (2008)
De, S., Smith, S.: Batch normalization biases residual blocks towards the identity function in deep networks. Adv. Neural. Inf. Process. Syst. 33, 19964–19975 (2020)
Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2015)
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
Lei, X., Jiang, X., Wang, C.: Design and implementation of a real-time video stream analysis system based on ffmpeg. In: 2013 Fourth World Congress on Software Engineering, pp. 212–216. IEEE (2013)
Li, J., Allinson, N.M.: A comprehensive review of current local features for computer vision. Neurocomputing 71(10–12), 1771–1787 (2008)
Mahmoudi, M., Sapiro, G.: Fast image and video denoising via nonlocal means of similar neighborhoods. IEEE Signal Process. Lett. 12(12), 839–842 (2005)
Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z.: Multi-class generative adversarial networks with the l2 loss function, vol. 5, p. 00102. arXiv preprint arXiv:1611.04076 (2016)
Martın, J., Jiménez, A., Seco, F., Calderón, L., Pons, J.L., Ceres, R.: Estimating the 3d-position from time delay data of us-waves: experimental analysis and a new processing algorithm. Sens. Actuators, A 101(3), 311–321 (2002)
Post, D.L., Calhoun, C.S.: An evaluation of methods for producing desired colors on crt monitors. Color Res. Appli. 14(4), 172–186 (1989)
Rieder, P., Scheffler, G.: New concepts on denoising and sharpening of video signals. IEEE Trans. Consum. Electron. 47(3), 666–671 (2001)
Serrano, R.S.: Deinterlacing algorithms. Albalá Ingenieros SA (2016)
Wang, J., Jeon, G., Jeong, J.: Efficient adaptive intra-field deinterlacing algorithm using bilateral filter. In: 2012 3rd IEEE International Conference on Network Infrastructure and Digital Content, pp. 468–472. IEEE (2012)
Xue, T., Chen, B., Wu, J., Wei, D., Freeman, W.T.: Video enhancement with task-oriented flow. Int. J. Comput. Vision 127(8), 1106–1125 (2019)
Zhu, H., Liu, X., Mao, X., Wong, T.T.: Real-time deep video deinterlacing. arXiv preprint arXiv:1708.00187 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Song, C., Li, H., Zheng, D., Wang, J., Jiang, Z., Yang, B. (2024). Transformer-Based Video Deinterlacing Method. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14451. Springer, Singapore. https://doi.org/10.1007/978-981-99-8073-4_28
Download citation
DOI: https://doi.org/10.1007/978-981-99-8073-4_28
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8072-7
Online ISBN: 978-981-99-8073-4
eBook Packages: Computer ScienceComputer Science (R0)