Rong Wang
ABSTRACT
The latest video coding standard, Versatile Video Coding (VVC) also called H.266, adopts quadtree with nested multitype tree (QTMT) coding block structure and still uses recursive rate-distortion (RD) to find the best intraframe coding unit (CU) partition, which accounts for more than 97% of the coding time. To address this problem, in this paper, we propose a fast CNN-based intraframe CU partition algorithm. The proposed approach, containing the CNN model, is implemented on the VVC official software VTM 14.0. A CU size of 64 × 64 is the input of the CNN, and the common test condition (CTC) sequences are encoded using four QPs {22,27,32,37} in the All-Intra (AI) configuration to verify the proposed method. The results showed that the encoding time was reduced by 42.15% in AI mode at the cost of a 0.85% increase in Bjøntegaard delta bit rate (BD-BR) compared with the official VTM14.0.
- Pakdaman, F., Adelimanesh, M. A., Gabbouj, M., & Hashemi, M. R. 2020, October. Complexity analysis of next-generation VVC encoding and decoding. In 2020 IEEE International Conference on Image Processing (ICIP) (pp. 3134-3138). IEEE.Google Scholar
Cross Ref
- Tissier, A., Mercat, A., Amestoy, T., Hamidouche, W., Vanne, J., & Menard, D. 2019, September. Complexity reduction opportunities in the future VVC intra encoder. In 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP) (pp. 1-6). IEEE.Google ScholarCross Ref
- Saldanha, M., Sanchez, G., Marcon, C., & Agostini, L. 2020, October. Complexity analysis of VVC intra coding. In 2020 IEEE International Conference on Image Processing (ICIP) (pp. 3119-3123). IEEE.Google ScholarCross Ref
- Shen, L., Liu, Z., Zhang, X., Zhao, W., & Zhang, Z. 2012. An effective CU size decision method for HEVC encoders. IEEE transactions on multimedia, 15(2), 465-470.Google Scholar
- Fu, B., Zhang, Q., & Hu, J. 2020. Fast prediction mode selection and CU partition for HEVC intra coding. IET Image Processing, 14 (9), 1892-1900.Google ScholarCross Ref
- Li, T., Xu, M., & Deng, X. 2017, July. A deep convolutional neural network approach for complexity reduction on intra-mode HEVC. In 2017 IEEE International Conference on Multimedia and Expo (ICME) (pp. 1255-1260). IEEE.Google ScholarCross Ref
- Kim, K., & Ro, W. W. 2018. Fast CU depth decision for HEVC using neural networks. IEEE Transactions on Circuits and Systems for Video Technology, 29(5), 1462-1473.Google ScholarDigital Library
- Xu, M., Li, T., Wang, Z., Deng, X., Yang, R., & Guan, Z. 2018. Reducing complexity of HEVC: A deep learning approach. IEEE Transactions on Image Processing, 27(10), 5044-5059.Google ScholarCross Ref
- Li, T., Xu, M., Deng, X., & Shen, L. 2020. Accelerate CTU partition to real time for HEVC encoding with complexity control. IEEE Transactions on Image Processing, 29, 7482-7496.Google ScholarDigital Library
- Feng, A., Gao, C., Li, L., Liu, D., & Wu, F. 2021, July. Cnn-Based Depth Map Prediction for Fast Block Partitioning in HEVC Intra Coding. In 2021 IEEE International Conference on Multimedia and Expo (ICME) (pp. 1-6). IEEE.Google Scholar
- Tao, H., Lu, Jin., & Li, Y. 2021. A Fast Division Algorithm for VVC Intra-Frame Coding Units. Journal of Chinese Mini-Micro Computer Systems (07), 1470-1474. doi:Google Scholar
- Cui, J., Zhang, T., Gu, C., Zhang, X., & Ma, S. 2020, March. Gradient-Based Early Termination of CU Partition in VVC Intra Coding. In DCC (pp. 103-112).Google Scholar
- Fu, T., Zhang, H., Mu, F., & Chen, H. 2019, July. Fast CU partitioning algorithm for H. 266/VVC intra-frame coding. In 2019 IEEE International Conference on Multimedia and Expo (ICME) (pp. 55-60). IEEE.Google ScholarCross Ref
- Yang, H., Shen, L., Dong, X., Ding, Q., An, P., & Jiang, G. 2019. Low-complexity CTU partition structure decision and fast intra mode decision for versatile video coding. IEEE Transactions on Circuits and Systems for Video Technology, 30(6), 1668-1682.Google ScholarCross Ref
- Galpin, F., Racapé, F., Jaiswal, S., Bordes, P., Le Léannec, F., & François, E. 2019, March. CNN-based driving of block partitioning for intra slices encoding. In 2019 Data Compression Conference (DCC) (pp. 162-171). IEEE.Google ScholarCross Ref
- Tissier, A., Hamidouche, W., Vanne, J., Galpin, F., & Menard, D. 2020, October. CNN oriented complexity reduction of VVC intra encoder. In 2020 IEEE International Conference on Image Processing (ICIP) (pp. 3139-3143). IEEE.Google ScholarCross Ref
- Zhang, Q., Guo, R., Jiang, B., & Su, R. 2021. Fast CU Decision-Making Algorithm Based on DenseNet Network for VVC. IEEE Access, 9, 119289-119297.Google ScholarCross Ref
- Li, T., Xu, M., Tang, R., Chen, Y., & Xing, Q. 2021. DeepQTMT: A deep learning approach for fast QTMT-based CU partition of intra-mode VVC. IEEE Transactions on Image Processing, 30, 5377-5390.Google ScholarCross Ref
- Fu, P. C., Yen, C. C., Yang, N. C., & Wang, J. S. 2021, June. Two-phase Scheme for Trimming QTMT CU Partition using Multi-branch Convolutional Neural Networks. In 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS) (pp. 1-6). IEEE.Google Scholar
- S.-h. Park and J.-W. Kang, "Fast Multi-Type Tree Partitioning for Versatile Video Coding Using a Lightweight Neural Network," in IEEE Transactions on Multimedia, vol. 23, pp. 4388-4399, 2021, doi: 10.1109/TMM.2020.3042062.Google Scholar
- Yeo, W. H., & Kim, B. G. 2021. CNN-based Fast Split Mode Decision Algorithm for Versatile Video Coding (VVC) Inter Prediction. Journal of Multimedia Information System, 8(3), 147-158.Google ScholarCross Ref
- Zhu, X., and M. Bain. "B-CNN: Branch Convolutional Neural Network for Hierarchical Classification." 2017.Google Scholar
- He, K., Zhang, X., Ren, S., & Sun, J. 2016, October. Identity mappings in deep residual networks. In European conference on computer vision (pp. 630-645). Springer, Cham.Google Scholar
Recommendations
CU encoding depth prediction, early CU splitting termination and fast mode decision for fast HEVC intra-coding
High Efficiency Video Coding (HEVC) is a new video coding standard achieving about a 50% bit rate reduction compared to the popular H.264/AVC High Profile with the same subjective reproduced video quality. Better coding efficiency is attained, however, ...
An early split and skip algorithm for fast intra CU selection in HEVC
The rapid development and increase of multimedia applications, as well as the demand for higher video-quality services at restricted resources such as storage capacity, transmission bandwidth and power consumption, has brought the urgent need for more ...
Fast Intra-Mode and CU Size Decision for HEVC
The latest video coding standard High Efficiency Video Coding (HEVC) achieves about a 50% bit-rate reduction compared with H.264/AVC under the same perceptual video quality. For intra coding, a coding unit (CU) is recursively divided into a ...
Comments