Skip to main content
SpringerLink
Log in

Efficient multiview video plus depth coding for 3D-HEVC based on complexity classification of the treeblock

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

High efficiency video coding standard-based 3D video (3D-HEVC) has been extended from HEVC to improve the coding efficiency of multiview video plus depth (MVD). Similar to the joint model of HEVC, a computationally expensive exhaustive mode decision is performed to find the least rate-distortion cost for each treeblock in 3D-HEVC. Furthermore, additional coding tools have been added to 3D-HEVC for improving the coding efficiency of the dependent texture video and depth map. Those tools achieve the highest possible coding efficiency, but also bring a significant computational complexity which limits 3D-HEVC from real-time applications. In order to reduce computational complexity, we propose an efficient multiview video plus depth coding algorithm for 3D-HEVC that adaptively utilizes the complexity classification of the treeblock. The coding complexity model of a treeblock is first analyzed according to the prediction mode and coding mode from the corresponding treeblocks in the reference views. Based on the complexity classification model of the treeblock, we propose two efficient low-complexity approaches, including fast mode size decision and adaptive motion search range selection. Extensive experimental results demonstrate that the proposed MVD coding algorithm can achieve the average computational saving about 60.1% with negligible rate-distortion performance loss in comparison with the original 3D-HEVC encoder.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Urey, H., Chellappan, K.V., Erden, E., Surman, P.: State of the art in stereoscopic and autostereoscopic displays. Proc. IEEE 99(4), 540–555 (2011)

    Article  Google Scholar 

  2. Chen, Y., Vetro, A.: Next-generation 3D formats with depth map support. IEEE MultiMed. 21(2), 90–94 (2014)

    Article  Google Scholar 

  3. Fehn, C.: Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV. Proc. SPIE Stereosc. Disp. Virt. Real. Syst. XI 5291, 93–104 (2004)

  4. Tech, G., Chen, Y., Müller, K., Ohm, J., Vetro, A.: Overview of the multiview and 3D extensions of high efficiency video coding. IEEE Trans. Circuits Syst. Video Technol. 26(1), 35–49 (2016)

    Article  Google Scholar 

  5. Müller, K., Schwarz, H., Marpe, D., Bartnik, C., Bosse, S., Brust, H., Hinz, T., Lakshman, H., Merkle, P., Rhee, H., Tech, G., Winken, M., Wiegand, T.: 3D high efficiency video coding for multi-view video and depth data. IEEE Trans. Circuits Syst. Video Technol. 22(9), 3366–3378 (2013)

    MathSciNet  MATH  Google Scholar 

  6. Sullivan, G.J., Boyce, J.M., Ying, C., Ohm, J.-R., Segall, C.A., Vetro, A.: Standardized extensions of high efficiency video coding (HEVC). IEEE J. Sel. Top. Signal Process. 7(6), 1001–1016 (2013)

    Article  Google Scholar 

  7. Shen, L., Liu, Z., An, P., Ma, R., Zhang, Z.: Low-Complexity mode decision for MVC. IEEE Trans. Circuits Syst. Video Technol. 6(21), 837–843 (2011)

    Article  Google Scholar 

  8. Zhao, T., Kwong, S., Wang, H., Wang, Z., Pan, Z., Kuo, C.-C.J.: Multiview coding mode decision with hybrid optimal stopping model. IEEE Trans. Image Process. 22(4), 1598–1609 (2013)

    Article  MathSciNet  Google Scholar 

  9. Yeh, C.H., Li, M.F., Chen, M.J., Chi, M.C., Huang, X.X., Chi, H.W.: Fast mode decision algorithm through inter-view rate-distortion prediction for multiview video coding system. IEEE Trans. Ind. Inform. 10(1), 594–603 (2014)

    Article  Google Scholar 

  10. Lee, J.Y., Wey, H.-C., Park, D.-S.: A fast and efficient multi-view depth image coding method based on temporal and inter-view correlations of texture images. IEEE Trans. Circuits Syst. Video Technol. 21(12), 1859–1868 (2011)

    Article  Google Scholar 

  11. Shen, L., An, P., Liu, Z., Zhang, Z.: Low complexity depth coding assisted by coding information from color video. IEEE Trans. Broadcast. 60(1), 128–133 (2014)

    Article  Google Scholar 

  12. Lei, J., Sun, J., Pan, Z., Kwong, S., Duan, J., Hou, C.: Fast mode decision using inter-view and inter-component correlations for multiview depth video coding. IEEE Trans. Ind. Inform. 11(4), 978–986 (2015)

    Article  Google Scholar 

  13. Zhang, Q., An, P., Zhang, Y., Shen, L., Zhang, Z.: Low complexity multiview video plus depth coding. IEEE Trans. Consum. Electron. 57(4), 1857–1865 (2011)

    Article  Google Scholar 

  14. Tohidypour, H.R., Pourazad, M.T., Nasiopoulos, P.: A low complexity mode decision approach for HEVC-based 3D video coding using a Bayesian method. In: Proceedings 2014 IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP), pp. 895–899 (2014)

  15. Zhang, N., Zhao, D., Chen, Y., Lin, J., Gao, W.: Fast encoder decision for texture coding in 3D-HEVC. Signal Process. Image Commun. 29(9), 951–961 (2014)

    Article  Google Scholar 

  16. Shen, L., An, P., Zhang, Z., Hu, Q., Chen, Z.: A 3D-HEVC fast mode decision algorithm for real-time applications. ACM Trans. Multimed. Comput. Commun. Appl. 11(3), 34 (2015)

    Article  Google Scholar 

  17. Park, C.: Edge-Based Intramode Selection for Depth-Map Coding in 3D-HEVC. IEEE Trans. Image Process. 24(1), 155–162 (2015)

    Article  MathSciNet  Google Scholar 

  18. Zhang, H., Chan, Y., Fu, C., Tsang, S., Siu, W.: Quadtree decision for depth intra coding in 3D-HEVC by good feature. In: Proceedings 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1481–1485 (2016)

  19. da Silva, T.L., Agostini, L.V., da Silva Cruz, L.A.: Fast intra prediction algorithm based on texture analysis for 3D-HEVC encoders. J. Real-Time Image Process. 12(2), 357–368 (2016)

    Article  Google Scholar 

  20. Tohidypour, H.R., Pourazad, M.T., Nasiopoulos, P.: Online-learning-based complexity reduction scheme for 3D-HEVC. IEEE Trans. Circuits Syst. Video Technol. 26(10), 1870–1883 (2016)

    Article  Google Scholar 

  21. Lei, J., Duan, J., Wu, F., Ling, N., Hou, C.: Fast mode decision based on grayscale similarity and inter-view correlation for depth map coding in 3D-HEVC. IEEE Trans. Circuits Syst. Video Technol. (2016). doi:10.1109/TCSVT.2016.2617332

    Article  Google Scholar 

  22. Zhang, H., Fu, C., Chan, Y., Tsang, S., Siu, W.: Probability-based depth intra mode skipping strategy and novel VSO metric for DMM decision in 3D-HEVC. IEEE Trans. Circuits Syst. Video Technol. (2017). doi:10.1109/TCSVT.2016.2612693

    Article  Google Scholar 

  23. Amish, F., Bourennane, E.B.: An efficient hardware solution for 3D-HEVC intra-prediction. J Real-Time Image Process. (2017). doi:10.1007/s11554-016-0664-1

    Article  Google Scholar 

  24. Zhang, Q., Chang, H., Wu, Q., Gan, Y.: Fast motion and disparity estimation for HEVC based 3D video coding. Multidim. Syst. Signal Process. 27(3), 743–761 (2016)

    Article  Google Scholar 

  25. Shen, L., Zhang, Z., Liu, Z.: Effective CU Size decision for HEVC intracoding. IEEE Trans. Image Process. 23(10), 4232–4241 (2014)

    Article  MathSciNet  Google Scholar 

  26. Zhang, Q., Chang, H., Huang, X., Huang, L., Su, R., Gan, Y.: Adaptive early termination mode decision for 3D-HEVC using inter-view and spatio-temporal correlations. AEUE Int. J. Electron. Commun. 70(5), 727–737 (2016)

    Article  Google Scholar 

  27. Shen, L., Zhang, Z., Liu, Z.: Adaptive inter-mode decision for HEVC jointly utilizing inter-level and spatio-temporal correlations. IEEE Trans. Circuits Syst. Video Technol. 24(10), 1709–1722 (2014)

    Article  Google Scholar 

  28. Zhang, Q., Wang, X., Huang, X., Su, R., Gan, Y.: Fast mode decision algorithm for 3D-HEVC encoding optimization based on depth information. Digit. Signal Process. 44(9), 37–46 (2015)

    Article  Google Scholar 

  29. Zhang, L., Tech, G., Wegner, K., Yea, S.: Test Model 6 of 3D-HEVC and MV-HEVC. Joint Collaborative Team on 3D Video Coding Extensions (JCT-3V) document JCT3V-F1005, 6th Meeting: Geneva, Switzerland (2013)

  30. Mueller, K., Vetro, A.: Common test conditions of 3DV core experiments. Joint Collaborative Team on 3D Video Coding Extensions (JCT-3V) document JCT3V-G1100, 7th Meeting: San Jose, CA, USA, (2014)

  31. Tanimoto, M., Fujii, T., Suzuki, K.: View synthesis algorithm in view synthesis reference software 2.0 (VSRS 2.0). ISO/IEC JTC1/SC29/WG11 document M16090, Lausanne, Switzerland (2008)

  32. Bjontegaard, G.: Calculation of average PSNR difference between RD-curves. In Proceedings 13th VCEG-M33 Meeting, Austin, TX, USA (2001)

Download references

Acknowledgements

The authors would like to thank the editors and anonymous reviewers for their valuable comments. This work was supported in part by the National Natural Science Foundation of China under Grant No. 61302118, 61401404, 61501407, 61572445 and 61502435, the Program for Science and Technology Innovation Talents in Universities of Henan Province under Grant No.17HASTIT022, the Funding Scheme of Young Key Teacher of Henan Province Universities under Grant No. 2016GGJS-087, the Scientific and Technological Project of Henan Province under Grant No. 142300410248, and 162102210214, the Graduate Scientific Research Foundation of Zhengzhou University of Light Industry, the Scientific and Technological of the Education Department of Henan Province under Grant No. 17B510011,15A520033, 16A520030, 15A413006 and 16A520028 and in part by the Doctorate Research Funding of Zhengzhou University of Light Industry, under Grant No. 2013BSJJ047.

Author information

Authors and Affiliations

  1. College of Computer and Communication Engineering, Zhengzhou University of Light Industry, No. 5 Dongfeng Road, Zhengzhou, 450002, China

    Qiuwen Zhang, Kunqiang Huang, Xiao Wang, Bin Jiang & Yong Gan

Authors
  1. Qiuwen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kunqiang Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bin Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yong Gan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Q., Huang, K., Wang, X. et al. Efficient multiview video plus depth coding for 3D-HEVC based on complexity classification of the treeblock. J Real-Time Image Proc 16, 1909–1926 (2019). https://doi.org/10.1007/s11554-017-0692-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-017-0692-5

Keywords

Search

Navigation