Skip to main content
SpringerLink
Log in

Fast CU partition decision for H.266/VVC based on the improved DAG-SVM classifier model

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

One of the biggest changes in H.266/Versatile Video Coding (VVC) is introduced quad-tree with nested multi-type tree (QTMT) coding tree architecture, where the multi-type tree (MTT) structure in H.266/VVC includes binary tree (BT) and ternary tree (TT). Compared with H.265/High Efficiency Video Coding (HEVC) which only is divided by quad-tree (QT), the QTMT architecture makes the coding unit (CU) partition procedure more complexity. In this paper, we design a fast CU partition decision algorithm based on the improved Directed Acyclic Graph Support Vector Machine (DAG-SVM) model to reduce the complexity of CU partition. The video sequences are first encoded on the H.266/VVC and Test Model 4.0 (VTM 4.0), and the characteristics of the video sequences are extracted for training through the improved F-score method, where the correlation between a feature and CU partition is high. Then, the offline training is used for the improved DAG-SVM model. Finally, the trained DAG-SVM model is embedded in VTM 4.0 to early forecast the optimal CU partition modes. Simulation results indicate that the proposed method increases the time savings to 54.74% while maintaining the encoding performance. Furthermore, the proposed method exceeds the latest methods of H.266/VVC.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Chen, J., Karczewicz, M., Huang, Y., Choi, K., Ohm, J., Sullivan, G.J.: The joint exploration model (JEM) for video compression with capability beyond HEVC. IEEE Trans. Circuits Syst. Video Technol. 30(5), 1208–1225 (2020)

    Article  Google Scholar 

  2. Bossen, F., Li, X., Sühring, K.: AHG report: test model software development (AHG3). Document JVET-Q0003 of ITU-T/ISO/IEC Joint Video Exploration Team (JVET), 17th meeting, Brussels, BE (2020)

  3. Yan, C., Shao, B., Zhao, H., Ning, R., Zhang, Y., Xu, F.: 3D room layout estimation from a single rgb image. IEEE Trans. Multimedia (2020). https://doi.org/10.1109/TMM.2020.2967645

  4. Chen, J., Ye, Y., Kim, S.H.: Algorithm description for versatile video coding and test model 8 (VTM 8). Document JVET-Q2002 of ITU-T/ISO/IEC Joint Video Exploration Team (JVET), 17th meeting, Brussels, BE (2020)

  5. Correa, G., Assuncao, P.A., Agostini, L.V., da Silva Cruz, L.A.: Fast HEVC encoding decisions using data mining. IEEE Trans. Circuits Syst. Video Technol. 25(4), 660–673 (2015)

    Article  Google Scholar 

  6. de Olievira, J.F., Alencar, M.S.: Online learning early skip decision method for the HEVC inter process using the SVM-based Pegasos algorithm. Electron. Lett. 52(14), 1227–1229 (2016)

    Article  Google Scholar 

  7. Shen, L., Zhang, Z., An, P.: Fast CU size decision and mode decision algorithm for HEVC intra coding. IEEE Trans. Consum. Electron. 59(1), 207–213 (2013)

    Article  Google Scholar 

  8. Zhang, Y., Wang, H., Li Z.: Fast coding unit depth decision algorithm for inter frame coding in HEVC. In: Proceedings of 2013 Data Compression Conference, Snowbird, UT, pp. 53–62 (2013). https://doi.org/10.1109/DCC.2013.13

  9. Shen, L., Zhang, Z., Liu, Z.: Effective CU size decision for HEVC intra coding. IEEE Trans. Image Process. 23(10), 4232–4241 (2014)

    Article  MathSciNet  Google Scholar 

  10. Min, B., Cheung, R.C.C.: A fast CU size decision algorithm for the HEVC intra encoder. IEEE Trans. Circuits Syst. Video Technol. 25(5), 892–896 (2015)

    Article  Google Scholar 

  11. Lee, J., Kim, S., Lim, K., Lee, S.: A fast CU size decision algorithm for HEVC. IEEE Trans. Circuits Syst. Video Technol. 25(3), 411–421 (2015)

    Article  Google Scholar 

  12. Kim, H.-S., Park, R.-H.: Fast CU partitioning algorithm for HEVC using an online-learning-based Bayesian decision rule. IEEE Trans. Circuits Syst. Video Technol. 26(1), 130–138 (2016)

    Article  Google Scholar 

  13. Shen, X., Yu, Y.: CU splitting early termination based on weighted SVM. Eurasip J. Image Video Process 2013(4), 1–11 (2013)

    Google Scholar 

  14. Zhang, Y., Kwong, S., Wang, X., Yuan, H., Pan, Z., Xu, L.: Machine learning-based coding unit depth decisions for flexible complexity allocation in High Efficiency Video Coding. IEEE Trans. Image Process. 24(7), 2225–2238 (2015)

    Article  MathSciNet  Google Scholar 

  15. Luo, Y., Liang, F., Xie, L.: A fast CU coding mode decision algorithm for H.265/HEVC. TENCON 2015 - 2015 IEEE Region 10 Conference, Macao, pp. 1–4 (2015). https://doi.org/10.1109/TENCON.2015.7372747

  16. Gao, X., Li, Z., Zhang, Y.: A fast HEVC inter CU size decision algorithm based on multi-class learning. In: Proceedings of 2018 10th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), Hangzhou, vol. 2, pp. 64–68 (2018). https://doi.org/10.1109/IHMSC.2018.10121

  17. Sun, C., Fan, X., Zhao, D.: A fast intra CU size decision algorithm based on canny operator and SVM classifier. In: Proceedings of 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, pp. 1787–1791 (2018). https://doi.org/10.1109/ICIP.2018.8451178

  18. Chen, K., Zeng, X., Fan, Y.: CNN oriented fast CU partition decision and PU mode decision for HEVC intra encoding. In: Proceedings of 2018 14th IEEE International Conference on Solid-State and Integrated Circuit Technology (ICSICT), Qingdao, pp. 1–3 (2018). https://doi.org/10.1109/ICSICT.2018.8564981

  19. Xu, Y., Yu, C., Lin, Y.: Fast Intra size decision and mode decision algorithm for HEVC intra coding. In: Proceedings of 2019 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (PACRIM), Victoria, BC, Canada, pp. 1–6 (2019). https://doi.org/10.1109/PACRIM47961.2019.8985078

  20. Liu, X., Li, Y., Liu, D., Wang, P., Yang, L.T.: An Adaptive CU Size Decision Algorithm for HEVC Intra Prediction Based on Complexity Classification Using Machine Learning. IEEE Trans. Circuits Syst. Video Technol. 29(1), 144–155 (2019)

    Article  Google Scholar 

  21. Liu, Z., Yu, X., Gao, Y., Chen, S., Ji, X., Wang, D.: CU Partition Mode Decision for HEVC Hardwired Intra Encoder Using Convolution Neural Network. IEEE Trans. Image Process. 25(11), 5088–5103 (2016)

    Article  MathSciNet  Google Scholar 

  22. Lin, Y. C., Wu, J.J., Chen, K.H.: CU partition prediction scheme for X265 intra coding using neural networks. In: Proceedings of 2019 4th International Conference on Control, Robotics and Cybernetics (CRC), Tokyo, Japan, pp. 206–210 (2019). https://doi.org/10.1109/CRC.2019.00049

  23. Zhang, Y., Pan, Z., Li, N., Wang, X., Jiang, G., Kwong, S.: Effective Data Driven Coding Unit Size Decision Approaches for HEVC INTRA Coding. IEEE Trans. Circuits Syst. Video Technol. 28(11), 3208–3222 (2018)

    Article  Google Scholar 

  24. Grellert, M., Zatt, B., Bampi, S., da Silva Cruz, L.A.: Fast coding unit partition decision for HEVC using support vector machines. IEEE Trans. Circuits Syst. Video Technol. 29(6), 1741–1753 (2019). https://doi.org/10.1109/TCSVT.2018.2849941

    Article  Google Scholar 

  25. Li, K., Wang, J.: Fast CU-splitting decisions based on data mining. In: Proceedings of 2016 IEEE International Conference on Consumer Electronics-China (ICCE-China), Guangzhou, pp. 1–5, (2016). https://doi.org/10.1109/ICCE-China.2016.7849745

  26. Xu, Y., Huang, X.; Hardware-oriented fast CU size and prediction mode decision algorithm for HEVC intra prediction. In: Proceedings of 2019 IEEE 5th International Conference for Convergence in Technology (I2CT), Bombay, India, pp. 1–5 (2019). https://doi.org/10.1109/I2CT45611.2019.9033606

  27. Cristina, O. C., Mihnea, U. R., Ionut, P.: HEVC intra partitioning and mode decision using histograms of oriented gradients. In: Proceedings of 2016 12th IEEE International Symposium on Electronics and Telecommunications (ISETC), Timisoara, pp. 277–280 (2016)

  28. Jin, Z., An, P., Shen, L., Yang, C.: CNN oriented fast QTBT partition algorithm for JVET intra coding. In: Proceedings of 2017 IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, pp. 1–4 (2017). https://doi.org/10.1109/VCIP.2017.8305020

  29. Jin, Z., An, P., Yang, C., Shen, L.: Fast QTBT partition algorithm for intra frame coding through convolutional neural network. IEEE Access 6, 54660–54673 (2018)

    Article  Google Scholar 

  30. Lin, T., Jiang, H., Huang, J., Chang P.: Fast binary tree partition decision in H.266/FVC intra Coding. In: Proceedings of 2018 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), Taichung, pp. 1–2 (2018). https://doi.org/10.1109/ICCE-China.2018.8448619

  31. Fu, T., Zhang, H., Mu, F., Chen, H.: Fast CU partitioning algorithm for H.266/VVC intra-frame coding. In: Proceedings of 2019 IEEE International Conference on Multimedia and Expo (ICME), Shanghai, China, pp. 55–60, (2019)

  32. Chen, J., Chiu, Y., Lee C., Tsai, Y.: Utilize neighboring LCU depth information to speedup FVC/H.266 intra coding. In: Proceedings of 2019 International Conference on System Science and Engineering (ICSSE), Dong Hoi, Vietnam, pp. 308–312 (2019). https://doi.org/10.1109/ICSSE.2019.8823436

  33. Park, S., Kang, J.: Context-based ternary tree decision method in versatile video coding for fast intra coding. IEEE Access 7, 172597–172605 (2019)

    Article  Google Scholar 

  34. Chen, J., Sun, H., Katto, J., Zeng, X., Fan, Y.: Fast QTMT partition decision algorithm in VVC intra coding based on variance and gradient. In: Proceedings of 2019 IEEE Visual Communications and Image Processing (VCIP), Sydney, Australia, pp. 1–4 (2019). https://doi.org/10.1109/VCIP47243.2019.8965674

  35. Tang, G., Jing, M., Zeng, X., Fan, Y.: Adaptive CU split decision with pooling-variable CNN for VVC intra encoding. In: Proceedings of 2019 IEEE Visual Communications and Image Processing (VCIP), Sydney, Australia, pp. 1–4 (2019). https://doi.org/10.1109/VCIP47243.2019.8965679

  36. Amestoy, T., Mercat, A., Hamidouche, W., Menard, D., Bergeron, C.: Tunable VVC frame partitioning based on lightweight machine learning. IEEE Trans. Image Process. 29(1), 1313–1328 (2020)

    Article  MathSciNet  Google Scholar 

  37. Tang, N., Cao, J., Liang, F., Wang, J., Liu, H., Wang, X., Du, X.: Fast CTU partition decision algorithm for VVC intra and inter Coding. In: Proceedings of 2019 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS), Bangkok, Thailand, pp. 361–364 (2019)

  38. Yang, H., Shen, L., Dong, X., Ding, Q., An, P., Jiang, G.: Low complexity CTU partition structure decision and fast intra mode decision for versatile video coding. IEEE Trans. Circuits Syst. Video Technol. 30(6), 1668–1682 (2020)

    Article  Google Scholar 

  39. Yan, C., Li, Z., Zhang, Y., Liu, Y., Ji, X., Zhang, Y.: Depth image denoising using nuclear norm and learning graph model. ACM Trans. Multimedia Comp. Comm. Appl. (2020)

  40. Yan, C., Gong, B., Wei, Y., Gao, Y.: Deep multi-view enhancement hashing for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. (2020). https://doi.org/10.1109/TPAMI.2020.2975798

    Article  Google Scholar 

  41. Bjontegaard, G.: Calculation of average PSNR differences between RD curves. In: Proceedings of 13th VCEG-M33 Meeting, Austin, TX, USA (2001)

  42. Methodology for the subjective assessment of the quality of television pictures. Int. Telecommun. Union-Radiocommun. Sect. ITR-R BT.500-11 (2002)

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China No. 61771432, 61302118, and 61702464.

Author information

Authors and Affiliations

  1. College of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou, 450002, China

    Qiuwen Zhang, Yihan Wang, Lixun Huang, Bin Jiang & Xiao Wang

Authors
  1. Qiuwen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yihan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lixun Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bin Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiuwen Zhang.

Additional information

Communicated by Y. Zhang.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Q., Wang, Y., Huang, L. et al. Fast CU partition decision for H.266/VVC based on the improved DAG-SVM classifier model. Multimedia Systems 27, 1–14 (2021). https://doi.org/10.1007/s00530-020-00688-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-020-00688-z

Keywords

Search

Navigation