Skip to main content
Springer Nature Link
Log in

Fast depth intra mode decision using intra prediction cost and probability in 3D-HEVC

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

3D-HEVC, which is a HEVC-compatible 3D video coding standard, was mainly developed to efficiently compress both a texture image and a depth map. Since the characteristics of the depth map are drastically different from those of the texture image, many advanced tools were adopted for depth intra coding in 3D-HEVC. In particular, a depth modelling mode (DMM) is evaluated to accurately predict sharp edges between objects. As a result, encoding complexity becomes very high. In order to reduce the high complexity, a fast depth intra mode decision method employing intra prediction cost and probability is proposed in this paper. Based on the cost and probability, the proposed method adaptively skips HEVC prediction modes and DMM in the mode decision. Experimental results demonstrate that it significantly reduces the encoding complexity, compared to conventional methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. “Cisco Visual Networking Index: Forecast and Methodology, 2016–2021,” (http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/complete-white-paper-c11-481360.html). Accessed 22 Dec 2022

  2. Wiegand T, Sullivan GJ, Bjntegaard G, Luthra A (2013) Overview of the H.264/AVC Video Coding Standard. IEEE Trans Circuits Syst Video Technol 13(7):560–576

    Article  Google Scholar 

  3. Sullivan GJ, Ohm J-R, Han W-J, Wiegand T (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Trans Circuits Syst Video Technol 22(12):1649–1668

    Article  Google Scholar 

  4. Bross B, Wang Y-K, Ye Y, Liu S, Chen J, Sullivan GJ, Ohm J-R (2021) Overview of the versatile video coding (VVC) standard and its applications. IEEE Trans Circuits Syst Video Technol 31(10):3736–3764

    Article  Google Scholar 

  5. Zhang Y, Reinhard E, Bull D (2011) Perception-based high dynamic range video compression with optimal bit-depth transformation. 2011 18th IEEE International Conference on Image Processing. Brussels, Belgium, pp 1321–1324

  6. Zhang Y, Naccari M, Agrafiotis D, Mrak M, Bull D (2016) High dynamic range video compression exploiting luminance masking. IEEE Trans Circuits Syst Video Technol 26(5):950–964

    Article  Google Scholar 

  7. François E, Segall CA, Tourapis AM, Yin P, Rusanovskyy D (2020) High dynamic range video coding technology in responses to the joint call for proposals on video compression with capability beyond HEVC. IEEE Trans Circuits Syst Video Technol 30(5):1253–1266

    Article  Google Scholar 

  8. Wu J, Yuen C, Cheung N-M, Chen J, Chen CW (2015) Enabling adaptive high-frame-rate video streaming in mobile cloud gaming applications. IEEE Trans Circuits Syst Video Technol 25(12):1998–2001

    Article  Google Scholar 

  9. Hou X, Dey S, Zhang J, Budagavi M (2020) Predictive adaptive streaming to enable mobile 360-degree and VR experiences. IEEE Trans on Multimedia 23:716–731

    Article  Google Scholar 

  10. Wei X, Zhou M, Kwong S, Yuan H, Jia W (2022) A hybrid control scheme for 360-degree dynamic adaptive video streaming over mobile devices. IEEE Trans on Mobile Comput 21(10):3428–3442

    Article  Google Scholar 

  11. Wang X, Shao F, Jiang Q, Fu Z, Meng X, Gu K, Ho Y-S (2021) Combining retargeting quality and depth perception measures for quality evaluation of retargeted stereopairs. IEEE Trans on Multimedia 24:2422–2434

    Article  Google Scholar 

  12. Wan W, Huang D, Shang B, Wei S, Wu HR, Wu J, Shi G (2023) Depth perception assessment of 3D videos based on stereoscopic and spatial orientation structural features. IEEE Trans Circuits Syst Video Technol 33(9):4588–4602

    Article  Google Scholar 

  13. Lafruit G, Bonatto D, Tulvan C, Preda M, Yu L (2019) Understanding MPEG-I coding standardization in immersive VR/AR applications. SMPTE Motion Imaging J 128(10):33–39

    Article  Google Scholar 

  14. Lee JY, Park HW (2020) HEVC-based three-layer texture and depth coding for lossless synthesis in 3D video coding. Multimed Tools Appl 79(29–30):20929–20945

    Article  Google Scholar 

  15. Le TV, Choi Y, Lee JY (2023) Specular detection and rendering for immersive multimedia. IEEE MultiMedia 30(2):38–47

    Article  Google Scholar 

  16. Choy S-M, Cheng E, Wilkinson RH, Burnett I, Austin MW (2021) quality of experience comparison of stereoscopic 3D videos in different projection devices: flat screen, panoramic screen and virtual reality Headset. IEEE Access 9:9584–9594

    Article  Google Scholar 

  17. Friston S, Griffith E, Swapp D, Lrondi C, Jjunju F, Ward R, Marshall A, Steed A (2021) Quality of service impact on edge physics simulations for VR. IEEE Trans Visual Comput Graphics 27(25):2694–2701

    Google Scholar 

  18. Lee JY, Lin J-L, Chen Y-W, Chang Y-L, Kovliga I, Fartukov A, Mishurosvskiy M, Wey H-C, Huang Y-W, Lei S (2015) Depth-based texture coding in AVC-compatible 3d video coding. IEEE Trans Circuits Syst Video Technol 25(8):1347–1361

    Article  Google Scholar 

  19. Sullivan GJ, Boyce JM, Chen Y, Ohm J-R, Segall CA, Vetro A (2013) Standardized extensions of high efficiency video coding (HEVC). IEEE J Sel Top Signal Process 7(6):1001–1016

    Article  Google Scholar 

  20. Fehn C (2004) Depth-Image-Based Rendering (DIBR), compression, and transmission for a new approach on 3D-TV." Stereoscopic displays and virtual reality systems XI. vol 5291. SPIE

  21. Lee JY, Park HW (2016) Efficient synthesis-based depth map coding in AVC-Compatible 3D Video Coding. IEEE Trans Circuits Syst Video Technol 26(6):1107–1116

    Article  Google Scholar 

  22. Lee JY, Park MW, Kim C (2015) 3D-CE1: Depth Intra Skip (DIS) Mode,” ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 Doc. JCT3V-K0033

  23. Muller K, Merkle P, Tech, Tech G, Wiegand T (2012) 3D video coding with depth modeling modes and view synthesis optimization. APSIPA Annual Summit and Conference

  24. Liu H, Chen Y (2014) Generic Segment-Wise DC for 3D-HEVC Depth Intra Coding. IEEE International Conference on Image Processing (ICIP), Paris, France, pp 3219–3222. https://doi.org/10.1109/ICIP.2014.7025651

  25. Ortega A, Ranchandra K (1998) Rate-distortion methods for image and video compression. IEEE Signal Process Mag 15(6):23–50

    Article  Google Scholar 

  26. Sanchez G, Saldanha M, Balota G, Zatt B, Porto M, Agostini L (2014) A complexity reduction algorithm for depth maps intra prediction on the 3D-HEVC. 2014 IEEE Vis Commun Image Proc Conf, Valletta, Malta, pp 137–140

  27. Park C-S (2015) Edge-based intramode selection for depth-map coding in 3D-HEVC. IEEE Trans Image Process 24(1):155–162

    Article  MathSciNet  Google Scholar 

  28. Zhang Q, Yang Y, Chang H, Zhang W, Gan Y (2017) Fast Intra Mode Decision for Depth Coding in 3D-HEVC. Multidim Syst Signal Process 28(4):1023–1226

    Article  MathSciNet  Google Scholar 

  29. Lee JY (2019) Fast depth intra mode decision based on mode analysis in 3D video coding. Electronics 8(4):430

    Article  Google Scholar 

  30. Fu C-H, Chen H, Chan Y-L, Tsang S-H, Zhu X (2020) Early termination for fast intra mode decision in depth map coding using dis-inheritance. Signal Proces: Image Commun 80:115644

    Google Scholar 

  31. Yao W, Wang XX, Yang D, Li W (2021) A fast wedgelet partitioning for depth map prediction in 3D-HEVC. Proc. SPIE 11720, Twelfth International Conference on Graphics and Image Processing (ICGIP 2020) 117200X

  32. Lainema J, Bossen F, Han W-J, Min J, Ugur K (2022) Intra Coding of the HEVC Standard. IEEE Trans on Circuit Syst Video Technol 22(12):1792–1801

    Article  Google Scholar 

  33. Müller K, Vetro A (2014) Common Test Conditions of 3DV Core Experiments. ISO/IEC and ITU-T, Doc. JCT3V-G1100

  34. Bjontegaard G (2001) Calculation of Average PSNR Differences between RD-Curves. ITU-T, Doc. VCEG-M33

  35. Kim H-S, Park R-H (2016) Fast CU partitioning algorithm for HEVC using an online-learning-based bayesian decision rule. IEEE Trans Circuit Syst Video Technol 26(1):130–138

    Article  Google Scholar 

  36. Zhang T, Sun M-T, Zhao D, Gao W (2017) Fast intra-mode and CU size decision for HEVC. IEEE Trans Circuits Syst Video Technol 27(8):1714–1726

    Article  Google Scholar 

  37. Liu X, Li Y, Liu D, Wang P, Yang LT (2019) An adaptive CU size decision algorithm for HEVC intra prediction based on complexity classification using machine learning. IEEE Trans Circuits Syst Video Technol 29(1):144–155

    Article  Google Scholar 

  38. Yang H, Shen L, Dong X, Ding Q, An P, Jiang G (2021) Low-complexity CTU partition structure decision and fast intra mode decision for versatile video coding. IEEE Trans Circuits Syst Video Technol 30(6):1668–16682

    Article  Google Scholar 

  39. Choi K, Le TV, Choi Y, Lee JY (2022) Low-complexity intra coding in versatile video coding. IEEE Trans Consumer Electronics 68(2):119–126

    Article  Google Scholar 

  40. Sole J, Joshi R, Nguyen N, Ji T, Karczewicz M, Clare G, Henry F, Duenas A (2012) Transform coefficient coding in HEVC. IEEE Trans Circuits Syst Video Technol 22(12):1765–1777

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in prat by the Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (IITP-2024-RS-2022-00156345), in part by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (RS-2023-00219051), and in part by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (RS-2022-00167169).

Author information

Authors and Affiliations

  1. Department of Artificial Intelligence and Robotics, Sejong University, Seoul, Republic of Korea

    Jin Young Lee

  2. School of Computer Science and Engineering, Kyungpook National University, Daegu, Republic of Korea

    Sang-hyo Park

Authors
  1. Jin Young Lee
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Sang-hyo Park
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Sang-hyo Park.

Ethics declarations

Competing of interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, J.Y., Park, Sh. Fast depth intra mode decision using intra prediction cost and probability in 3D-HEVC. Multimed Tools Appl 83, 80411–80424 (2024). https://doi.org/10.1007/s11042-024-18794-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-024-18794-9

Keywords