Skip to main content
SpringerLink
Log in

Fast video encoding based on random forests

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Machine learning approaches have been increasingly used to reduce the high computational complexity of high-efficiency video coding (HEVC), as this is a major limiting factor for real-time implementations, due to the decision process required to find optimal coding modes and partition sizes for the quad-tree data structures defined by the standard. This paper proposes a systematic approach to reduce the computational complexity of HEVC based on an ensemble of online and offline Random Forests classifiers. A reduced set of features for training the Random Forests classifier is proposed, based on the rankings obtained from information gain and a wrapper-based approach. The best model parameters are also obtained through a consistent and generalizable method. The proposed Random Forests classifier is used to model the coding unit and transform unit-splitting decision and the SKIP-mode prediction, as binary classification problems, taking advantage from the combination of online and offline approaches, which adapts better to the dynamic characteristics of video content. Experimental results show that, on average, the proposed approach reduces the computational complexity of HEVC by 62.64% for the random access (RA) profile and 54.57% for the low-delay (LD) main profile, with an increase in BD-Rate of 2.58% for RA and 2.97% for LD, respectively. These results outperform the previous works also using ensemble classifiers for the same purpose.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Sullivan, G.J., Ohm, J.R., Han, W.J., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circ. Syst. Video Technol. 22(12), 1649 (2012)

    Google Scholar 

  2. Correa, G., Assuncao, P., Agostini, L., Da Silva Cruz, L.A.: Performance and computational complexity assessment of high-efficiency video encoders. IEEE Trans. Circ. Syst. Video Technol. 22(12), 1899 (2012)

    Google Scholar 

  3. Shen, L., Liu, Z., Zhang, X., Zhao, W., Zhang, Z.: An effective CU size decision method for HEVC encoders. IEEE Trans. Multimedia 15(2), 465 (2013)

    Google Scholar 

  4. Lee, Hoyoung, Shim, Huik Jae, Park, Younghyeon, Jeon, B.: Early skip mode decision for HEVC encoder with emphasis on coding quality. IEEE Trans. Broadcast. 61(3), 388 (2015)

    Google Scholar 

  5. Zhang, Y., Kwong, S., Wang, X., Yuan, H., Pan, Z., Xu, L.: Machine learning based coding unit depth decisions for flexible complexity allocation in high efficiency video coding. IEEE Trans. Image Process. 24(7), 2225 (2015)

    MathSciNet  MATH  Google Scholar 

  6. Correa, G., Assuncao, P., Agostini, L., Da Silva Cruz, L.A.: Fast HEVC encoding decisions using data mining. IEEE Trans. Circ. Syst. Video Technol. 25(4), 660 (2015)

    Google Scholar 

  7. Breiman, L.: Random forests. J. Mach. Learn. 45(1), 5 (2001)

    MATH  Google Scholar 

  8. Rhee, C.E., Lee, K., Kim, T., Lee, H.J.: A survey of fast mode decision algorithms for inter-prediction and their applications to high efficiency video coding. IEEE Trans. Consum. Electron. 58(4), 1375 (2012)

    Google Scholar 

  9. Sun, X., Chen, X., Xu, Y., Xiao, Y., Wang, Y., Yu, D.: Fast CU size and prediction mode decision algorithm for HEVC based on direction variance. J. Real-Time Image Proc. (2017). https://doi.org/10.1007/s11554-017-0682-7

    Article  Google Scholar 

  10. Shen, L., Zhang, Z., Liu, Z.: Adaptive inter-mode decision for HEVC jointly utilizing inter-level and spatiotemporal correlations. IEEE Trans. Circuits Syst. Video Technol. 24(10), 1709 (2014)

    Google Scholar 

  11. Lin, T.L., Chou, C.C., Liu, Z., Tung, K.H.: HEVC early termination methods for optimal CU decision utilizing encoding residual information. J. Real-Time Image Proc. (2016). https://doi.org/10.1007/s11554-016-0608-9

    Article  Google Scholar 

  12. Tai, Kh, Hsieh, My, Chen, Mj, Chen, Cy, Yeh, C.H.: A fast HEVC encoding method using depth information of collocated CUs and RD cost characteristics of PU modes. IEEE Trans. Broadcast. 63(4), 680 (2017)

    Google Scholar 

  13. Chen, F., Li, P., Peng, Z., Jiang, G., Yu, M., Shao, F.: A fast inter coding algorithm for hevc based on texture and motion quad-tree models. Signal Process. Image Commun. 47, 271 (2016)

    Google Scholar 

  14. Huang, X., Zhang, Q., Zhao, X., Zhang, W., Zhang, Y., Gan, Y.: Fast inter-prediction mode decision algorithm for HEVC. Signal Image Video Process. 11(1), 33 (2017)

    Google Scholar 

  15. Jaja, E.T., Omar, Z., Ab Rahman, A.AH., et al.: Enhanced inter-mode decision algorithm for HEVC/H. 265 video coding. J. Real-Time Image Proc. (2015). https://doi.org/10.1007/s11554-015-0542-2

    Article  Google Scholar 

  16. Ahn, S., Lee, B., Kim, M.: A novel fast CU encoding scheme based on spatiotemporal encoding parameters for HEVC inter coding. IEEE Trans. Circ. Syst. Video Technol. 25(3), 422 (2015)

    Google Scholar 

  17. Lee, J.H., Goswami, K., Kim, B.G., Jeong, S., Choi, J.S.: Fast encoding algorithm for high-efficiency video coding (HEVC) system based on spatio-temporal correlation. J. Real-Time Image Process. 12(2), 407 (2016)

    Google Scholar 

  18. Shen, X., Yu, L., Chen, J.: Fast coding unit size selection for HEVC based on Bayesian decision rule. In: Picture Coding Symposium (PCS), 2012 (IEEE, 2012), pp. 453–456

  19. Shen, L., Zhang, Z., Zhang, X., An, P., Liu, Z.: Fast TU size decision algorithm for HEVC encoders using Bayesian theorem detection. Signal Process. Image Commun. 32, 1–8 (2015)

    Google Scholar 

  20. Xiong, J., Li, H., Wu, Q., Meng, F.: A fast HEVC inter CU selection method based on pyramid motion divergence. IEEE Trans. Multimedia 16(2), 559 (2014)

    Google Scholar 

  21. Grellert, M., Zatt, B., Bampi, S., da Silva Cruz, L.A.: Fast coding unit partition decision for HEVC using support vector machines. IEEE Trans. Circ. Syst. Video Technol. (2018). https://doi.org/10.1109/TCSVT.2018.2849941

    Article  Google Scholar 

  22. Kim, H.S., Park, R.H.: Fast CU partitioning algorithm for HEVC using an online-learning-based bayesian decision rule. IEEE Trans. Circ. Syst. Video Technol. 26(1), 130 (2016)

    Google Scholar 

  23. Zhu, L., Zhang, Y., Pan, Z., Wang, R., Kwong, S., Peng, Z.: Binary and multi-class learning based low complexity optimization for HEVC encoding. IEEE Trans. Broadcast. 63(3), 547 (2017)

    Google Scholar 

  24. Shen, X., Yu, L.: CU splitting early termination based on weighted SVM. EURASIP J. Image Video Process. 2013(4), 1 (2013)

    Google Scholar 

  25. Ruiz, D., Fernández-Escribano, G., Martínez, J.L., Cuenca, P.: A unified architecture for fast HEVC intra-prediction coding. J. Real-Time Image Proc. (2017). https://doi.org/10.1007/s11554-017-0685-4

    Article  Google Scholar 

  26. Du, B., Siu, W.C., Yang, X.: Fast CU partition strategy for HEVC intra-frame coding using learning approach via random forests. In 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2015 (0), 1085 (2016)

  27. Woźniak, M., Graña, M., Corchado, E.: A survey of multiple classifier systems as hybrid systems. Inf. Fusion 16, 3 (2014)

    Google Scholar 

  28. Fern, M., Cernadas, E.: Do we need hundreds of classifiers to solve real world classification problems ? J. Mach. Learn. Res. 15(1), 3133 (2014)

    MathSciNet  MATH  Google Scholar 

  29. Duda, R.O., Hart, P.E.P.E., Stork, D.G.: Pattern Classification. Wiley, Oxford (2001)

    MATH  Google Scholar 

  30. Yu, L., Liu, H.: Efficient feature selection via analysis of relevance and redundancy. J. Mach. Learn. Res. 5, 1205 (2004)

    MathSciNet  MATH  Google Scholar 

  31. Hall, M.A., Holmes, G.: Benchmarking attribute selection techniques for discrete class data mining. IEEE Trans. Knowl. Data Eng. 15(6), 1437 (2003)

    Google Scholar 

  32. Zhu, L., Zhang, Y., Li, N., Jiang, G., Kwong, S.: Machine learning based fast h.264/avc to hevc transcoding exploiting block partition similarity. J. Vis. Commun. Image Rep. 38, 824 (2016)

    Google Scholar 

  33. Fawcett, T.: Tom: an introduction to ROC analysis. Pattern Recogn. Lett. 27(8), 861 (2006)

    MathSciNet  Google Scholar 

  34. Mallikarachchi, T., Talagala, D.S., Arachchi, H.K., Fernando, A.: Content-adaptive feature-based CU size prediction for fast low-delay video encoding in HEVC. IEEE Trans. Circ. Syst. Video Technol. 8215(c), 1 (2016)

    Google Scholar 

  35. High Efficiency Video Coding (HEVC) | JCT-VC. https://hevc.hhi.fraunhofer.de/. Accessed 2 Feb 2019

  36. Lee, B.N.: librf: C++ random forests library. http://mtv.ece.ucsb.edu/benlee/librf.html. Accessed 2 Feb 2019

  37. Bossen, F.: Common test conditions and software reference configurations. in 12th Meeting of JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 (Geneva)

  38. Bjontegaard, G.: Calculation of average PSNR differences between RD-curves. In ITU - T SG16 Q. 6 VCEG-M33 (Austin, Texas)

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for their comments and suggestions to improve this work. Pedro A. Assuncao would like to acknowledge the support of Fundacao para a Ciencia e Tecnologia (FCT) by Instituto de Telecomunicacoes (IT), grant UID/EEA/50008/2013, and Project ARoundVision SAICT-45-2017-POCI-01-0145-FEDER-030652, PTDC/EEI-COM/30652/2017, Portugal.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Capital University of Science and Technology, Islamabad, Pakistan

    Muhammad Tahir & Imtiaz A. Taj

  2. Polytechnic Institute of Leiria and Instituto de Telecomunicacoes (IT), Morro do Lena-Alto do Vieiro, 2411-901, Leiria, Portugal

    Pedro A. Assuncao

  3. Department of Computer Science, Lahore Garrison University, Lahore, Pakistan

    Muhammad Asif

Authors
  1. Muhammad Tahir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Imtiaz A. Taj
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pedro A. Assuncao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Muhammad Asif
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Tahir.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tahir, M., Taj, I.A., Assuncao, P.A. et al. Fast video encoding based on random forests. J Real-Time Image Proc 17, 1029–1049 (2020). https://doi.org/10.1007/s11554-019-00854-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-019-00854-1

Keywords

Search

Navigation