Skip to main content
SpringerLink
Log in

Optimization of depth modeling modes in 3D-HEVC depth intra coding

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

In the development of 3D video extension of high efficiency video coding (HEVC) standard, known as 3D-HEVC, depth modeling modes (DMMs) are introduced in depth intra coding to represent object edges in depth maps. With the DMMs, a depth block is approximated by partitioning the block into two non-rectangular regions using Wedgelet or Contour partition, where each region is represented by a constant value referred to as a constant partition value (CPV). To predict the CPV more accurately and efficiently, we develop three approaches in this study. First, a better CPV predictor may be obtained by simply extending the actual depth map boundary, which can also simplify the CPV prediction by removing comparisons and average operations. Second, we propose to choose an optimal combination of delta CPVs in terms of view synthesis optimization at the encoder by checking more candidates. Finally, zero residual coding is suggested for DMMs coding units in the rate-distortion optimization loop. Experimental results demonstrate that about 0.2 and 0.1 % Bjøntegaard delta rate saving can be achieved on average for synthesized views with less complexity, under the all-intra and random access configurations, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. ISO/IEC JTC1/SC29/WG11: Applications and requirements on 3D video coding. Doc. N12035, Geneva, Switzerland (2011)

  2. ISO/IEC JTC1/SC29/WG11: Call for proposals on 3D video coding technology. Doc. N12036, Geneva (2011)

  3. Sullivan, G.J., Ohm, J.-R., Han, W.-J., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012)

    Article  Google Scholar 

  4. Müller, K., Schwarz, H., Marpe, D., Bartnik, C., et al.: 3D high-efficiency video coding for multi-view video and depth data. IEEE Trans. Image Process. 22(9), 3366–3378 (2013)

    Article  MathSciNet  Google Scholar 

  5. Zhu, C., Zhao, Y., Yu, L., Tanimoto, M.: 3D-TV system with depth-image-based rendering: architectures, techniques and challenges. Springer, Berlin (2013)

    Book  Google Scholar 

  6. Fehn, C.: Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV. Proceedings of SPIE conference on stereoscopic displays and virtual reality systems XI, Vol. 5291, pp. 93–104, CA, USA, Jan 2004

  7. Zhao, Y., Chen, Z., Tian, D., Zhu, C., Yu, L.: Suppressing texture-depth misalignment for boundary noise removal in view synthesis. 28th Picture Coding Symposium (PCS 2010), Nagoya, Japan, Dec 2010

  8. Lei, J., Li, S., Zhu, C., Sun, M.-T., Hou, C.: Depth coding based on depth-texture motion and structure similarities. IEEE Trans. Circuits Syst. Video Technol. 25(2) (2015)

  9. Li, S., Lei, J., Zhu, C., Yu, L., Hou, C.: Pixel-based inter prediction in coded texture assisted depth coding. IEEE Signal Process. Lett. 21(1), 74–78 (2014)

    Article  Google Scholar 

  10. Müller, K., Merkle, P., Tech, G., Wiegand, T.: 3D video coding with depth modeling modes and view synthesis optimization. Annual summit and conference of the asia-pacific-signal-and-information-processing-association (APSIPA), Hollywood, CA, Dec 2012

  11. Chen, Y., Tech, G., Wegner, K., Yea, S.: Test model 7 of 3D-HEVC and MV-HEVC[C]. ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JCT3V-J1003, 9th meeting: Strasbourg, FR, 2014

  12. Zhang, K., Chen, Y., An, J., Zhang, X., Lin, J., Lei, S.: 3D-CE3: results on simplified DMM mode coding. ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JCT3V-H0092, 8th meeting: Valencia, ES, 29 Mar–4 Apr 2014

  13. Chen, X., Zheng, X., Zhu, C., Lin, Y., Zheng, J.: 3D-CE3 related: redundancy removal for DMM1 signalling. ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JCT3V-H0110, 8th meeting: Valencia, ES, 29 Mar–4 Apr 2014

  14. Merkle, P., Marpe, D.: CE5 related: CE2 related: simplification of DMM4,” ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JCT3V-I0066, 9th meeting: Sapporo, JP, 3–9 July 2014

  15. Tsukuba, T., Ikai, T.: Lookup table size reduction in DMM1. ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JCT3V-I0110, 9th meeting: Sapporo, JP, 3–9 July 2014

  16. Ikai, T., Tsukuba, T.: Simplification of DMM table derivation. ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JCT3V-J0025, 10th meeting: Strasbourg, FR, 18–24 Oct 2014

  17. Sampaio, F., Zatt, B., Shafique, M., Agostini, L., Henkel, J., Bampi, S.: Content-adaptive reference frame compression based on intra-frame prediction for multiview video coding. 20th IEEE international conference on image processing (ICIP), Melbourne, Australia, Sep 2013

  18. Zatt, B., Shafique, M., Sampaio, F., Agostini, L., Bampi, S., Henkel, J.: Run-time adaptive energy-aware motion and disparity estimation in multiview video coding. ACM/EDAC/IEEE 48th design automation conference (DAC), pp. 1026–1031 (2011)

  19. Zatt, B., Shafique, M., Bampi, S., Henkel, J.: Multi-level pipelined parallel hardware architecture for high throughput motion and disparity estimation in multiview video coding. IEEE/ACM 14th design automation and test in europe conference (DATE), pp. 1448–1453 (2011)

  20. Shafique, M., Zatt, B., Walter, F.L., Bampi, S., Henkel, J.: Adaptive power management of on-chip video memory for multiview video coding. ACM/EDAC/IEEE 49th design automation conference (DAC), pp. 866–875 (2012)

  21. Sampaio, F., Shafique, M., Zatt, B., Bampi, S., Henkel, J.: dSVM: energy-efficient distributed scratchpad video memory architecture for the next-generation high efficiency video coding. IEEE/ACM 17th design automation and test in europe conference (DATE) (2014)

  22. Sampaio, F., Shafique, M., Zatt, B., Bampi, S., Henkel, J.: Content-driven memory pressure balancing and video memory power management for parallel high efficiency video coding. ACM/IEEE International Symposium on Low Power Electronics and Design (ISLPED), La Jolla, CA, USA, pp. 153–158 (2014)

  23. Khan, M.U.K., Shafique, M., Henkel, J.: Fast hierarchical intra angular mode selection for high efficiency video coding. 21st IEEE International Conference on Image Processing (ICIP) (2014)

  24. Jager, F.: Simplified depth map intra coding with an optional depth lookup table. International Conference on 3D Imaging (IC3D) (2012)

  25. Ma, S., Lin, Y., Zhu, C., Zheng, J., Yu, L., Wang, X.: Improved segment-wise DC coding for HEVC intra prediction of depth maps. APSIPA, pp. 1–4 (2014)

  26. Lee, J.Y., Park, M.W., Kim, C.: 3D-CE1: depth intra skip (DIS) mode. ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JCT3V-K0033, 11th meeting: Geneva, CH, 12–18 Feb 2015

  27. Zhao, X., Zhang, L., Chen, Y., Karczewicz, M.: CE6.h related: simplified DC predictor for depth intra. ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JCT3V-D0183, 4th meeting: Incheon, Korea, 20–26 Apr 2013

  28. Zhang, Y., Zhu, C., Lin, Y., Zheng, J., Wang, X.: Simplified reference pixel selection for constant partition value coding in 3D-HEVC. In Signal and Information Processing, 2015 IEEE China Summit and International Conference on, pp. 244–248, 12–15 July 2015

  29. Lee, J., Park, M., Kim, C.: 3D-CE5 related: simplified depth inter mode coding. Joint collaborative team on 3D video coding extensions (JCT-3V) document JCT3V-F0149, 6th meeting: Geneva, Switzerland, 25 Oct–1 Nov, 2013

  30. Lee, J.Y., Park, M.W., Cho, Y., Kim, C.: 3D-CE2 related: fast SDC DC offset decision. ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JCT3V-I0084, 9th meeting: Sapporo, JP, 3–9 July 2014

  31. HEVC 3D extension software repository (at HHI). [Online]. https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/tags/HTM-13.0/

  32. Mueller, K., Vetro, A.: Common test conditions of 3DV core experiments. ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JCT3V-G1100, 7th meeting: San José, US, 11–17 Jan 2014

  33. Bjontegaard, G.: Calculation of average PSNR differences between RD-curves. ITU-T Q.6/SG16 VCEG, VCEG-M33 (2001)

  34. HEVC 3D extension software repository (at HHI). [Online]. https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/tags/HTM-10.0r1/

Download references

Author information

Authors and Affiliations

  1. School of Electronic Engineering and the Center for Robotics, University of Electronic Science and Technology of China, Chengdu, 611731, Sichuan, China

    Yuhua Zhang, Yong Wang & Ce Zhu

  2. HiSilicon Technologies Co. Ltd., Beijing, China

    Yongbing Lin & Jianhua Zheng

Authors
  1. Yuhua Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ce Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yongbing Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianhua Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Wang, Y., Zhu, C. et al. Optimization of depth modeling modes in 3D-HEVC depth intra coding. J Real-Time Image Proc 13, 85–100 (2017). https://doi.org/10.1007/s11554-016-0589-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-016-0589-8

Keywords

Search

Navigation