Skip to main content
SpringerLink
Log in

Machine Learning-Based approaches to reduce HEVC intra coding unit partition decision complexity

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The use of machine learning techniques to reduce recent video coding standards complexity such as High Efficiency Video Coding (HEVC) has received prominent attention. In fact, the fascinating HEVC standard coding efficiency gap is performed at the expense of dramatically increasing coding complexity. HEVC adopts a similar block-based hybrid video coding framework its predecessor H.264 Advanced Video Coding (H.264/AVC), but it provides a highly flexible hierarchy of unit representation, which includes three units: coding unit (CU), prediction unit (PU) and transform unit (TU). The recursive splitting of CU is content adaptive, which is one of the biggest differences compared to H.264/AVC. Adopting a large variety of coding unit (CU) sizes, the quadtree partition takes the lion’s share of the HEVC encoding complexity, making it ever more challenging to use rigid traditional inference models to predict the efficient coding decisions. In this context, this paper investigates the resulting implications on both coding efficiency and encoding complexity, when using Fuzzy Support Vector Machine (FSVM) and convolutional Neural Network (CNN) models for partitioning in the HEVC intra-prediction. The first approach is an online FSVM-based algorithm designed to predict efficiently the CU partition module. The second one is a deep CNN method founded on a large-scale database of substantial CU partition training. Experimental results reveal that the proposed deep CNN approach, with 66.04% complexity reduction, outperforms the proposed online FSVM approach that achieves 45.33%. However, the FSVM with only 0.067% loss in coding efficiency compared to 1.69% engendered with the CNN, is considered as the approach that performs the best tradeoff between the compression efficiency and the complexity reduction when optimizing the HEVC complexity at all Intra configuration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, et al (2016) Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467

  2. Amna M, Imen W, Ezahra SF (2021) Lenet5-based approach for fast intra coding. In 2020 10th International Symposium on Signal, Image, Video and Communications (ISIVC), pp. 1–4

  3. Bjontegaard G (2001) Calculation of average psnr differences between rd-curves. VCEG-M33

  4. Bouaafia R, Khemiri S, Sayadi FE, Atri M (2020) Fast cu partition-based machine learning approach for reducing hevc complexity. J Real Time Image Process 17(1):185–196

    Article  Google Scholar 

  5. Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM transactions on intelligent systems and technology (TIST) 2(3):1–27

    Article  Google Scholar 

  6. Correa G, Assuncao P, Agostini L, L.A. eda SilvaCruz, (2012) Performance and computational complexity assessment of highefficiency video encoders. IEEE Transactions on Circuits and Systems for Video Technology 22(12):1899–1909

    Article  Google Scholar 

  7. Fernandez DG, Alberto A, Del B, Botella G, Meyer-Baese U, Meyer-Baese A, Grecos C (2017) Information fusion based techniques for hevc. In Real-Time Image and Video Processing  10223:102230M

  8. Fini MR, Zargari F (2019) Two stage fast mode decision algorithm for intra prediction. HEVC. Multimedia Tools and Applications 75(13):7541–7558

    Article  Google Scholar 

  9. Grellert M, Zatt B, Bampi S, da Silva Cruz LA (2018) Fast coding unit partition decision for HEVC using support vector machines. Trans Circuits Syst Video Technol. 29(6):1741–1753

    Article  Google Scholar 

  10. Hassan M, Shanableh T (2019) Predicting split decisions of coding units in hevc video compression using machine learning techniques. Multimedia Tools and Applications 78(23):32735–32754

    Article  Google Scholar 

  11. Hosseini E, Pakdaman F, Hashemi MR, Ghanbari M (2019) A computationally scalable fast intra coding scheme for hevc video encoder. Multimedia Tools and Applications 78(9):11607–11630

    Article  Google Scholar 

  12. JCT-VC (2019) Hm software. https://hevc.hhi.fraunhofer.de/. [Online; Accessed 15 Apr 2019]

  13. Jiang W, Ma H, Chen Y (2012) Gradient based fast mode decision algorithm for intra prediction in hevc. In 2012 2nd international conference on consumer electronics, communications and networks (CECNet), pp. 1836–1840

  14. Kuanar S, Rao KR, Conly C (2018) Fast mode decision in hevc intra prediction, using region wise cnn feature classification. In 2018 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1–4

  15. Lan C, Xu J, Sullivan GJ, Wu  F (2012) Intra transform skipping, document jctvc-i0408. Joint Collaborative Team on Video Coding (JCT-VC)

  16. Li T, Xu M, Deng X (2017) A deep convolutional neural network approach for complexity reduction on intra-mode hevc. In 2017 IEEE International Conference on Multimedia and Expo (ICME), pp. 1255–1260

  17. LI Z, Zhang Y, Li B (2012) Gradient-based fast decision for intra prediction in HEVC. In Vis Commun Image Process. pages 1–6

  18. Lin CF, Wang SD (2002) Fuzzy support vector machines. IEEE transactions on neural networks 13(2):464–471

    Article  Google Scholar 

  19. Liu X, Li Y, Liu D, Wang LT, Yang P (2017) An adaptive cu size decision algorithm for hevc intra prediction based on complexity classification using machine learning. IEEE Transactions on Circuits and Systems for Video Technology 29(1):144–155

    Article  Google Scholar 

  20. Liu Z, Yu X, Gao Y, Chen S, Ji X, Wang D (2016) Cu partition mode decision for hevc hardwired intra encoder using convolution neural network. IEEE Transactions on Image Processing 25(11):5088–5103

    Article  MathSciNet  Google Scholar 

  21. Mallikarachchi T, Fernando A,  Arachchi HK (2014) Efficient coding unit size selection based on texture analysis for hevc intra prediction. In 2014 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6

  22. Moraes D, Wainer J, Rocha A (2016) Low false positive learning with support vector machines. Journal of Visual Communication and Image Representation 38:340–350

    Article  Google Scholar 

  23. Shen X, YU L (2013) A fast hevc inter cu selection method based on pyramid motion divergence. EURASIP journal on image and video processing 2013(1):1–11

    Article  Google Scholar 

  24. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  25. Sullivan GJ, Ohm JR, Han WJ, Wiegand T (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Trans Circuits Syst Video Technol 22(12):1649–1668

    Article  Google Scholar 

  26. Tsai AC, Wang JF, Lin WG, Yang JF (2007) A simple and robust direction detection algorithm for fast h. 264 intra prediction. In 2007 IEEE International Conference on Multimedia and Expo, pp 1587–1590

  27. Wang C, Yu L, Wang S (2018) Accelerate cu partition in hevc using large-scale convolutional neural network. arXiv preprint arXiv:1809.08617

  28. Wiegand T, Sullivan GJ, Bjontegaard G, Luthra A (2003) Overview of the h. 264/avc video coding standard. IEEE Transactions on circuits and systems for video technology 13(7):560–576

    Article  Google Scholar 

  29. Xiph.org (2019) Xiph.org video test media. https://media.xiph.org/video/derf. [Online; Accessed 15 Apr 2019]

  30. Xiong J, Li H, Wu Q, Meng F (2013) A fast hevc inter cu selection method based on pyramid motion divergence. IEEE transactions on multimedia 16(2):559–564

    Article  Google Scholar 

  31. Xu M, Li T, Wang Z, Deng X, Yang Z, Guan R (2018) Reducing complexity of hevc: A deep learning approach. IEEE Transactions on Image Processing 27(10):5044–5059

    Article  MathSciNet  Google Scholar 

  32. Yin Y, Yang X, Lin J, Chen Y, Fang R (2018) A fast block partitioning algorithm based on svm for hevc intra coding. In Proceedings of the 2018 the 2nd International Conference on Video and Image Processing, pp. 176–181

  33. Zhang T, Sun MT, Zhao D, Gao W (2016) Fast intra-mode and cu size decision for hevc. IEEE Transactions on Circuits and Systems for Video Technology 27(8):1714–1726

    Article  Google Scholar 

  34. Zhang Y, Kwong A, Wang X, Yuan H, Pan Z, Xu L (2015) Machine learning-based coding unit depth decisions for flexible complexity allocation in high efficiency video coding. IEEE Transactions on Image Processing 24(7):2225–2238

    Article  MathSciNet  Google Scholar 

  35. Zhu L, Zhang Y, Pan Z, Wang R, Kwong S, Peng Z (2017) Binary and multi-class learning based low complexity optimization for hevc encoding. IEEE Transactions on Broadcasting 63(3):547–561

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electronics and Microelectronics Laboratory, Faculty of Sciences of Monastir, Environment Street, Monastir, 5019, Tunisia

    Maraoui Amna & Bouaafia Soulef

  2. Electronic and Information Technology Laboratory, University of Sfax, Sfax, Tunisia

    Werda Imen

  3. Networked objects, control and communication systems Laboratory, University of Sousse, Sousse, Tunisia

    Sayadi Fatma Ezahra

Authors
  1. Maraoui Amna
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Werda Imen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bouaafia Soulef
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sayadi Fatma Ezahra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maraoui Amna.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Amna, M., Imen, W., Soulef, B. et al. Machine Learning-Based approaches to reduce HEVC intra coding unit partition decision complexity. Multimed Tools Appl 81, 2777–2802 (2022). https://doi.org/10.1007/s11042-021-11678-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11678-2

Keywords

Search

Navigation