Abstract
Versatile Video Coding (VVC), the newest standard for future video coding, is currently under development. This proposal aimed to improve the encoder performance over the latest standard namely High Efficiency Video Coding, carried with a high increase in coding complexity. The VVC partition structure is mainly based on the quadtree with nested multi-type tree (QTMT) block scheme. Such an improvement leads to a more flexible block partition and promotes a high encoding efficiency, but generates a huge coding complexity. In order to deal with this issue, a fast QTMT intra partition algorithm, based on a deep neural network named Early Terminated Hierarchical Convolution Neural Network, is applied to predict the \(64\times \)64 block QT partition structure. The proposed algorithm determines the QTMT partition structure based on the decision of whether to split or skip the corresponding CU, in order to get \(128\times \)128 Coding Tree Unit partition architecture. In this paper, the proposed intra partition work achieves a significant speedup in encoding gain that reaches 32.96% in best cases for Ultra High Definition video sequences compared to the reference VVC software VTM-3.0. For all video sequences, 24.49% time saving is reached on average. This improvement comes with an increase of 4.18% and a decrease of 0.18 dB in terms of BDBR and BDPSNR, respectively.
Similar content being viewed by others
References
Akbari, A., Trocan, M., Granado, B.: Image compression using adaptive sparse representations over trained dictionaries. In: 2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP), IEEE, pp 1–6 (2016)
Amestoy, T., Mercat, A., Hamidouche, W., Bergeron, C., Menard, D.: Random forest oriented fast qtbt frame partitioning. ICASSP 2019–2019 IEEE International Conference on Acoustics, pp. 1837–1841. Speech and Signal Processing (ICASSP), IEEE (2019)
Bjøntegaard, G.: Calculation of average psnr differences between rd-curves (vceg-m33). In: VCEG Meeting (ITU-T SG16 Q. 6), pp 2–4 (2001)
Bossen, F., Boyce, J., Li, X., Seregin, V., Sühring, K.: Jvet common test conditions and software reference configurations for sdr video. Joint Video Experts Team (JVET) of ITU-T SG 16 (2018)
Cao, J., Tang, N., Wang, J., Liang, F.: Texture-based fast cu size decision and intra mode decision algorithm for vvc. In: International Conference on Multimedia Modeling, Springer, pp 739–751 (2020)
Grellert, M., Zatt, B., Bampi, S., da Silva Cruz, L.A.: Fast coding unit partition decision for hevc using support vector machines. IEEE Trans. Circuits Syst. Video Technol. 29(6), 1741–1753 (2018)
Jin, Z., An, P., Shen, L.: Fast qtbt partition algorithm for jvet intra coding based on cnn. In: Pacific Rim Conference on Multimedia, Springer, pp 59–69 (2017)
Liu, Z., Yu, X., Gao, Y., Chen, S., Ji, X., Wang, D.: Cu partition mode decision for hevc hardwired intra encoder using convolution neural network. IEEE Trans. Image Process. 25(11), 5088–5103 (2016)
Sidaty, N., Hamidouche, W., Deforges, O., Philippe, P.: Emerging video coding performance: 4k quality monitoring. In: 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX), IEEE, pp 1–3 (2017)
“VVC Test Model (VTM) vesion 30” (December 2018) [online] available. https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware VTM/tree/VTM-3.0
Xu, M., Li, T., Wang, Z., Deng, X., Yang, R., Guan, Z.: Reducing complexity of hevc: a deep learning approach. IEEE Trans. Image Process. 27(10), 5044–5059 (2018)
Yang, H., Shen, L., Dong, X., Ding, Q., An, P., Jiang, G.: Low complexity ctu partition structure decision and fast intra mode decision for versatile video coding. IEEE Trans. Circuits Syst. Video Technol. (2019)
Zhang, Y., Kwong, S., Wang, X., Yuan, H., Pan, Z., Xu, L.: Machine learning-based coding unit depth decisions for flexible complexity allocation in high efficiency video coding. IEEE Trans. Image Process. 24(7), 2225–2238 (2015)
Zhu, L., Zhang, Y., Pan, Z., Wang, R., Kwong, S., Peng, Z.: Binary and multi-class learning based low complexity optimization for hevc encoding. IEEE Trans. Broadcast. 63(3), 547–561 (2017)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Abdallah, B., Belghith, F., Ben Ayed, M.A. et al. Low-complexity QTMT partition based on deep neural network for Versatile Video Coding. SIViP 15, 1153–1160 (2021). https://doi.org/10.1007/s11760-020-01843-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-020-01843-9