Skip to main content
Springer Nature Link
Log in

Multi-level complexity reduction for HEVC multiview coding

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Standardized in 2014, multiview extension of high efficiency video coding (MV-HEVC) offers significantly better compression performance of up to 50% for multiview and 3D videos compared to multiple independent single view HEVC coding. However, the extreme high computational complexity of MV-HEVC demands significant optimization of the encoder. In this work, we propose a series of optimization techniques at various levels of abstraction: non-aggregation massively parallel motion estimation (ME) and disparity estimation (DE) for prediction units, fractional and bidirectional ME/DE, quantization parameter-based early termination of coding tree unit (CTU), and optimized resource-scheduled wave front parallel processing for CTU. When evaluated over three views for all available official multiview video coding test sequences, proposed optimization outperforms the anchor encoder by average factor of 5.4 at the cost of 4.4% bitrate (DBR) increase at no loss in PSNR, or alternatively a PSNR degradation of 0.12 dB at no change to the DBR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. The CPU on the platform used in the experiment (Table 3) supports up to Intel\(\circledR \) SSE4 SIMD instruction set with maximum register width of 128-bit.

References

  1. Bjontegaard, G.: Calculation of average PSNR differences between RD-curves. In: ITU—Telecommunications Standardization Sector Video Coding Expert Group (VCEG) 13th Meeting, ITU-T Study Group 16, Question 6 (ITU-T SG16 Q.6), Doc No. VCEG-M33, Austin, TX (2001)

  2. Bossen, F.: Common HM test conditions and software reference configurations. Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 8th Meeting, MPEG No. m24011, JCT-VC No. JCTVC-H1100 (2012)

  3. Chi, C.C., Alvarez-Mesa, M., Juurlink, B., Clare, G., Henry, F., Pateux, S., Schierl, T.: Parallel scalability and efficiency of hevc parallelization approaches. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1827–1838 (2012). https://doi.org/10.1109/TCSVT.2012.2223056

    Article  Google Scholar 

  4. Choi, K., Park, S.H., Jang, E.S.: Coding tree pruning based CU early termination. In: Document JCTVC-F092, JCT-VC, Torino, Italy (2011)

  5. Fan, R., Zhang, Y., Li, B.: Motion classification-based fast motion estimation for high efficiency video coding. IEEE Trans. Multimed. 99, 1–1 (2016). https://doi.org/10.1109/TMM.2016.2642786

    Article  Google Scholar 

  6. Flierl, M., Girod, B.: Multiview video compression. IEEE Signal Process. Mag. 24(6), 66–76 (2007). https://doi.org/10.1109/MSP.2007.905699

    Article  Google Scholar 

  7. Gweon, R.H., Lee, Y.L.: Early termination of CU encoding to reduce HEVC complexity. In: Document JCTVC-F045, JCT-VC, Torino, Italy (2011)

  8. Heinrich Hertz Institute (HHI) (2014) H.265/MPEG HEVC Multiview Coding Reference Software (HTM) 16.2. https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/

  9. ITU-T, JTC I (2012) Advanced video coding for generic audiovisual services

  10. ITU-T and ISO/IEC JTC (2015) H.265, Series H: Audiovisual and Multimedia Systems, Infrastructure of Audiovisual Services—Coding of Moving Video

  11. Jiang, C., Nooshabadi, S.: GPU accelerated motion and disparity estimations for multiview coding. In: IEEE International Conference on Image Processing (ICIP), Melbourne, Australia, pp. 2106–2110 (2013). https://doi.org/10.1109/ICIP.2013.6738434

  12. Jiang, C., Nooshabadi, S.: Parallel multiview video coding exploiting group of pictures level parallelism. IEEE Trans. Parallel Distrib. Syst. (2015). https://doi.org/10.1109/TPDS.2015.2485993

  13. Jiang, C., Nooshabadi, S.: A scalable massively parallel motion and disparity estimation scheme for multiview video coding. IEEE Trans. Circuits Syst. Video Technol. 26(2), 346–359 (2016)

    Article  Google Scholar 

  14. Jimnez-Moreno, A., Martnez-Enrquez, E., de Mara, F.D.: Complexity control based on a fast coding unit decision method in the HEVC video coding standard. IEEE Trans. Multimed. 18(4), 563–575 (2016). https://doi.org/10.1109/TMM.2016.2524995

    Article  Google Scholar 

  15. Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP3 and ISO/IEC JTC 1/SC 29/WG 11 (2015) HEVC Test Model (HM) 16. Reference Software. https://hevc.hhi.fraunhofer.de/

  16. Merkle, P., Smolic, A., Müller, K., Wiegand, T.: Efficient prediction structures for multiview video coding. IEEE Trans. Circuits Syst. Video Technol. 17(11), 1461–1473 (2007). https://doi.org/10.1109/TCSVT.2007.903665

    Article  Google Scholar 

  17. NVIDIA (2015) NVIDIA Compute Unified Device Architecture (CUDA) C Programming Guide 7.5. http://docs.nvidia.com/cuda/pdf/CUDA_C_Programming_Guide.pdf

  18. Pieters, B., Hollemeersch, C.F., J De Cock, P.L., Neve, W.D., Walle, R.V.D.: Parallel deblocking filtering in MPEG-4 AVC/H.264 on massively parallel architectures. IEEE Trans. Circuits Syst. Video Technol. 21(1), 96–100 (2011). https://doi.org/10.1109/TCSVT.2011.2105553

    Article  Google Scholar 

  19. Radicke, S., Hahn, J.U., Wang, Q., Grecos, C.: Many-Core HEVC Encoding Based on Wavefront Parallel Processing and GPU-Accelerated Motion Estimation. Springer, Cham, pp. 393–417 (2015). https://doi.org/10.1007/978-3-319-25915-4_21

  20. Rusanovsky, D., Müller, K., Vetro, A.: Common Test Conditions of 3DV Core Experiments. Joint Collaborative Team on 3D Video Coding Extension Development (JCT-3V) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 3rd Meeting, MPEG No. m28363, JCT3V-VC No. JCT3V-C1100 (2013)

  21. Shami, M., Hemani, A.: Classification of massively parallel computer architectures. In: IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) PhD Forum, 2012, pp. 344–351 (2012). https://doi.org/10.1109/IPDPSW.2012.42

  22. Sullivan, G., Boyce, J., Chen, Y., Ohm, J.R., Segall, C., Vetro, A.: Standardized extensions of high efficiency video coding (HEVC). IEEE J. Sel. Top. Signal Process. 7(6), 1001–1016 (2013). https://doi.org/10.1109/JSTSP.2013.2283657

    Article  Google Scholar 

  23. Sze, V., Budagavi, M. (eds.): GJS High Efficiency Video Coding Algorithms and Architectures. Springer, Cham (2014)

    Google Scholar 

  24. Tech, G., Chen, Y., Muller, K., Ohm, J.R., Vetro, A., Wang, Y.K.: Overview of the multiview and 3D extensions of high efficiency video coding. IEEE Trans. Circuits Syst. Video Technol. 26(1), 35–49 (2016). https://doi.org/10.1109/TCSVT.2015.2477935

    Article  Google Scholar 

  25. Tourapis, A.M.: Enhanced predictive zonal search for single and multiple frame motion estimation. In: Visual Communications and Image Processing, San Jose, CA, pp. 1069–1079 (2002)

  26. Vetro, A., Wiegand, T., Sullivan, G.: Overview of the stereo and multiview video coding extensions of the H.264/MPEG-4 AVC standard. Proc. IEEE 99(4), 626–642 (2011). https://doi.org/10.1109/JPROC.2010.2098830

    Article  Google Scholar 

  27. Wang, X., Song, L., Chen, M., Yang, J.: Paralleling variable block size motion estimation of HEVC on multi-core CPU plus GPU platform. In: IEEE International Conference on Image Processing (ICIP), Tain City, Taiwan, pp. 1836–1839 (2013). https://doi.org/10.1109/ICIP.2013.6738378

  28. Wu, B.F., Peng, H.Y., Yu, T.L.: Efficient hierarchical motion estimation algorithm and its vlsi architecture. IEEE Trans. Very Large Scale Integr. VLSI Syst. 16(10), 1385–1398 (2008). https://doi.org/10.1109/TVLSI.2008.2000526

    Article  Google Scholar 

  29. Xiao, W., Li, B., Xu, J., Shi, G., Wu, F.: HEVC encoding optimization using multicore CPUs and GPUs. IEEE Trans. Circuits Syst. Video Technol. 25(11), 1830–1843 (2015). https://doi.org/10.1109/TCSVT.2015.2406199

    Article  Google Scholar 

  30. Xie, L., Huang, L., Chen, B.: UMHexagonS search algorithm for fast motion estimation. In: IEEE International Conference on Computer Research and Development (ICCRD), Shanghai, China, vol. 1, pp. 483–487 (2011)

  31. Xiong, J., Li, H., Wu, Q., Meng, F.: A fast HEVC inter CU selection method based on pyramid motion divergence. IEEE Trans. Multimed. 16(2), 559–564 (2014). https://doi.org/10.1109/TMM.2013.2291958

    Article  Google Scholar 

  32. Yang, J., Kim, J., Won, K., Lee, H., Jeon, B.: Early SKIP detection for HEVC. In: document JCTVC-G543, JCT-VC, Geneva, Switzerland (2011)

  33. Zhao, L., Zhang, L., Ma, S., Zhao, D.: Fast mode decision algorithm for intra prediction in (hevc). In: IEEE Visual Communications and Image Processing (VCIP), pp. 1–4 (2011). https://doi.org/10.1109/VCIP.2011.6115979

  34. Zhu, C., Lin, X., Chau, L.: Hexagon-based search pattern for fast block motion estimation. IEEE Trans. Circuits Syst. Video Technol. 12(5), 349–355 (2002). https://doi.org/10.1109/TCSVT.2002.1003474

    Article  Google Scholar 

  35. Zhu, S., Ma, K.: A new diamond search algorithm for fast block-matching motion estimation. IEEE Trans. Image Process. 9(2), 287–290 (2000). https://doi.org/10.1109/83.821744

    Article  Google Scholar 

  36. Zone ID: Integer Intrinsic Intrinsics for Intel® Streaming SIMD Extensions 2 (Intel® SSE2) (2014). https://software.intel.com/en-us/node/583147

  37. Zupancic, I., Blasi, S.G., Peixoto, E., Izquierdo, E.: Inter-prediction optimizations for video coding using adaptive coding unit visiting order. IEEE Trans. Multimed. 18(9), 1677–1690 (2016). https://doi.org/10.1109/TMM.2016.2579505

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Michigan Tech, Houghton, MI, USA

    Caoyang Jiang & Saeid Nooshabadi

Authors
  1. Caoyang Jiang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Saeid Nooshabadi
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Saeid Nooshabadi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, C., Nooshabadi, S. Multi-level complexity reduction for HEVC multiview coding. J Real-Time Image Proc 17, 197–213 (2020). https://doi.org/10.1007/s11554-018-0757-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-018-0757-0

Keywords