Skip to main content
SpringerLink
Log in

Machine learning-based fast CU size decision algorithm for 3D-HEVC inter-coding

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

3D-high efficiency video coding (3D-HEVC) is an extension of the high efficiency video coding (HEVC) standard for the compression of the texture videos and depth maps. In 3D-HEVC inter-coding, the coding unit (CU) is recursively performed on variable sizes, namely, depth levels. The CU size decision process is conducted using all the possible depth levels to obtain the one with the least rate-distortion (RD) cost using the Lagrange multiplier. These tools achieve the highest coding efficiency but incur a very high computational complexity. In this paper, a fast CU size decision algorithm is proposed to reduce the complexity caused by the CU size splitting process. The proposed algorithm is based on the CU homogeneity classification using machine learning technology. First, the tensor feature is extracted to characterize the homogeneity of CU, which has a strong relationship with CU sizes. Then, a boosted decision stump algorithm is employed to analyze and construct a binary classification model from the extracted features and find suitable thresholds for the proposed method. Finally, an efficient early termination of CU splitting is released based on adaptive thresholds for texture videos and depth maps. The experimental results show that the proposed algorithm reduces a significant encoding time, while the loss in coding efficiency is negligible.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Müller, K., Merkle, P., Wiegand, T.: 3-D video representation using depth maps. Proc. IEEE 99, 643–656 (2011)

    Article  Google Scholar 

  2. Bosc, E., Pepion, R., Le Callet, P., Koppel, M., Ndjiki-Nya, P., Pressigout, M., Morin, L.: Towards a new quality metric for 3-D synthesized view assessment. IEEE J. Sel. Top. Signal Process. 5, 1332–1343 (2011)

    Article  Google Scholar 

  3. Smolic, A., Muller, K., Dix, K., Merkle, P., Kauff, P., Wiegand, T.: Intermediate view interpolation based on multiview video plus depth for advanced 3D video systems. In: 2008 15th IEEE International Conference on Image Processing (2008)

  4. Muller, K., Schwarz, H., Marpe, D., Bartnik, C., Bosse, S., Brust, H., Hinz, T., Lakshman, H., Merkle, P., Rhee, F., Tech, G., Winken, M., Wiegand, T.: 3D high-efficiency video coding for multi-view video and depth data. IEEE Trans. Image Process. 22, 3366–3378 (2013)

    Article  MathSciNet  Google Scholar 

  5. Tech, G., Chen, Y., Muller, K., Ohm, J., Vetro, A., Wang, Y.: Overview of the multiview and 3D extensions of high efficiency video coding. IEEE Trans. Circuits Syst. Video Technol. 26, 35–49 (2016)

    Article  Google Scholar 

  6. Jaja, E., Omar, Z., Ab Rahman, A., Zabidi, M.: Enhanced inter-mode decision algorithm for HEVC/H.265 video coding. J. Real-Time Image Process. 16, 377–390 (2015)

    Article  Google Scholar 

  7. Bouaafia, S., Khemiri, R., Sayadi, F., Atri, M.: Fast CU partition-based machine learning approach for reducing HEVC complexity. J. Real-Time Image Process. 17, 185–196 (2019)

    Article  Google Scholar 

  8. Lei, J., Duan, J., Wu, F., Ling, N., Hou, C.: Fast mode decision based on grayscale similarity and inter-view correlation for depth map coding in 3D-HEVC. IEEE Trans. Circuits Syst. Video Technol. 28, 706–718 (2018)

    Article  Google Scholar 

  9. Ahn, Y., Sim, D.: Square-type-first inter-CU tree search algorithm for acceleration of HEVC encoder. J. Real-Time Image Process. 12, 419–432 (2015)

    Article  Google Scholar 

  10. Bakkouri, S., Elyousfi, A., Hamout, H.: Fast CU size and mode decision algorithm for 3D-HEVC intercoding. Multimed. Tools Appl. 79, 6987–7004 (2019)

    Article  Google Scholar 

  11. Li, Y., Yang, G., Zhu, Y., Ding, X., Song, Y., Zhang, D.: Hybrid stopping model-based fast PU and CU decision for 3D-HEVC texture coding. J. Real-Time Image Process. (2019). https://doi.org/10.1007/s11554-019-00876-9

    Article  Google Scholar 

  12. Liao, Y., Chen, M., Yeh, C., Lin, J., Chen, C.: Efficient inter-prediction depth coding algorithm based on depth map segmentation for 3D-HEVC. Multimed. Tools Appl. 78, 10181–10205 (2018)

    Article  Google Scholar 

  13. Zhang, Q., Huang, K., Wang, X., Jiang, B., Gan, Y.: Efficient multiview video plus depth coding for 3D-HEVC based on complexity classification of the treeblock. J. Real-Time Image Process. 16, 1909–1926 (2017)

    Article  Google Scholar 

  14. Chen, J., Wang, B., Liao, J., Cai, C.: Fast 3D-HEVC inter mode decision algorithm based on the texture correlation of viewpoints. Multimed. Tools Appl. 78, 29291–29305 (2018)

    Article  Google Scholar 

  15. Zhang, Q., Zhang, N., Wei, T., Huang, K., Qian, X., Gan, Y.: Fast depth map mode decision based on depth-texture correlation and edge classification for 3D-HEVC. J. Vis. Commun. Image Represent. 45, 170–180 (2017)

    Article  Google Scholar 

  16. Chen, M., Yang, Y., Zhang, Q., Zhao, X., Huang, X., Gan, Y.: Low complexity depth mode decision for HEVC-based 3D video coding. Optik 127, 4758–4767 (2016)

    Article  Google Scholar 

  17. Sullivan, G., Ohm, J., Han, W., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22, 1649–1668 (2012)

    Article  Google Scholar 

  18. Sullivan, G., Boyce, J., Chen, Y., Ohm, J., Segall, C., Vetro, A.: Standardized extensions of high efficiency video coding (HEVC). IEEE J. Sel. Top. Signal Process. 7, 1001–1016 (2013)

    Article  Google Scholar 

  19. Mora, E., Jung, J., Cagnazzo, M., Pesquet-Popescu, B.: Initialization, limitation, and predictive coding of the depth and texture quadtree in 3D-HEVC. IEEE Trans. Circuits Syst. Video Technol. 24, 1554–1565 (2014)

    Article  Google Scholar 

  20. Joint Collaborative Team on 3D video coding (JCT-3V) HTM 16.2 Reference Software: [online]. https://www.hevc.hhi.fraunhofer.de/trac/3d-hevc/browser/3DVCSoftware/tags/HTM-16.2. Accessed 27 May 2016

  21. Mueller, K., Vetro, A.: Common test conditions of 3DV core experiments. In: Joint Collaborative Team on 3D Video Coding Extensions (JCT-3V) document JCT3V-G1100, 7th Meeting: San Jose, CA, USA, (2014)

  22. Safavian, S., Landgrebe, D.: A survey of decision tree classifier methodology. IEEE Trans. Syst. Man Cybern. 21, 660–674 (1991)

    Article  MathSciNet  Google Scholar 

  23. Espejo, P., Ventura, S., Herrera, F.: A survey on the application of genetic programming to classification. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 40, 121–144 (2010)

    Article  Google Scholar 

  24. Webb, G., Fürnkranz, J., Fürnkranz, J., Fürnkranz, J., Hinton, G., Sammut, C., Sander, J., Vlachos, M., Teh, Y., Yang, Y., Mladeni, D., Brank, J., Grobelnik, M., Zhao, Y., Karypis, G., Craw, S., Puterman, M., Patrick, J.: Decision Stump. In: Encyclopedia of Machine Learning, pp. 262–263 (2011). https://doi.org/10.1007/978-0-387-30164-8_202

  25. Schapire, R.: The Boosting Approach to Machine Learning: An Overview. In: Nonlinear Estimation and Classification, pp. 149–171 (2003). https://doi.org/10.1007/978-0-387-21579-2_9

  26. Setayesh, M., Zhang, M., Johnston, M.: A new homogeneity-based approach to edge detection using PSO. In: 2009 24th International Conference Image and Vision Computing New Zealand (2009)

  27. Saha, P., Udupa, J.: Optimum image thresholding via class uncertainty and region homogeneity. IEEE Trans. Pattern Anal. Mach. Intell. 23, 689–706 (2001)

    Article  Google Scholar 

  28. Chakraborty, A., Staib, L., Duncan, J.: Deformable boundary finding in medical images by integrating gradient and region information. IEEE Trans. Med. Imaging 15, 859–870 (1996)

    Article  Google Scholar 

  29. Bakkouri, S., Elyousfi, A.: Effective CU size decision algorithm based on depth map homogeneity for 3D-HEVC inter-coding. In: 2020 International Conference on Intelligent Systems and Computer Vision (ISCV) (2020)

  30. Baghaie, A., Yu, Z.: Structure tensor based image interpolation method. AEU Int J Electron Commun. 69, 515–522 (2015)

    Article  Google Scholar 

  31. Faraklioti, M., Petrou, M.: The use of structure tensors in the analysis of seismic data. In: Mathematics in Industry, pp. 47–88 (2005). https://doi.org/10.1007/3-540-26493-0_3

  32. Bjntegaard, G.: Calculation of average PSNR differences between RD curves. In: 13th VCEG Meeting, Document VCEGM33, Austin (2001)

  33. Bjntegaard, G.: Improvements of the BD-PSNR model. In: 35th VCEG Meeting, Document VCEG-AI11, Berlin (2008)

  34. Tanimoto, M., Fujii, T., Suzuki, K.: View synthesis algorithm in view synthesis reference software 2.0 (VSRS2.0). Technical report, ISO/IEC JTC1/SC29/WG11 M16090, Lausanne, Switzerland (2008)

Download references

Author information

Authors and Affiliations

  1. Computer Systems and Vision Laboratory, Faculty of Sciences, Ibn-Zohr University Agadir-Morocco, Agadir, Morocco

    Siham Bakkouri

  2. Department of Computer Science, National Engineering School of Applied Sciences, Ibn-Zohr University Agadir-Morocco, Agadir, Morocco

    Abderrahmane Elyousfi

Authors
  1. Siham Bakkouri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abderrahmane Elyousfi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Siham Bakkouri.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bakkouri, S., Elyousfi, A. Machine learning-based fast CU size decision algorithm for 3D-HEVC inter-coding. J Real-Time Image Proc 18, 983–995 (2021). https://doi.org/10.1007/s11554-020-01059-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-01059-7

Keywords

Search

Navigation