Skip to main content
SpringerLink
Log in

Quality-power configurable flexible coding order hardware design for real-time 3D-HEVC intra-frame prediction

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The emerging of 3D video-capable embedded mobile devices is expected due to the popularization of multimedia services and the demand for novel immersive video technologies. Such devices require efficient hardware-friendly heuristics to deal with strict processing requirements and limited energy supply. To contribute to these requirements, this work presents a complete 3D-HEVC intra-frame prediction hardware design that supports a flexible coding order between texture and depth channels. The developed hardware employs hardware-friendly constraints and novel heuristics to explore inter-channel redundancies and to reduce the computational effort through the novel inter-channel directional structure detector heuristic. The designed 3D-HEVC intra-frame prediction system dissipates 384.6 mW while processing three HD 1080p views (texture + depth) at 30 frames per second in real-time. To the best of our knowledge, this is the first work to propose a complete 3D-HEVC intra-frame prediction system with support to flexible coding order. In addition, this is the only hardware design to process luminance and chrominance texture channels and depth channel.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Pereira, F., Silva, E.: Efficient plenoptic imaging representation: why do we need it? IEEE Int. Conf. Multimed. Expo (ICME) (2016). https://doi.org/10.1109/ICME.2016.7552939

    Article  Google Scholar 

  2. Tech, G., Chen, Y., Müller, K., Ohm, J., Vetro, A., Wang, Y.: Overview of the multiview and 3D extensions of high efficiency video coding. IEEE Trans. Circuits Syst. Video Technol. 26(1), 35–49 (2016). https://doi.org/10.1109/TCSVT.2015.2477935

    Article  Google Scholar 

  3. Sullivan, G., Ohm, J., Han, W., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012). https://doi.org/10.1109/TCSVT.2012.2221191

    Article  Google Scholar 

  4. ITU-T.: Recommendation ITU-T H.265: high efficiency video coding (2015)

  5. ITU-T.: Recommendation ITU-T H.264: advanced video coding for generic audiovisual services (2017)

  6. Vetro, A., Pandit, P., Kimata, H., Smolic, A., Wang, Y-K.: Joint draft 8.0 on multiview video coding. Doc. JVT-AB204, Hannover (2008)

  7. Kauff, P., Atzpadin, N., Fehn, C., Müller, M., Schreer, O., Smolic, A., Tanger, R.: Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability. Signal Process. Image Commun. 22(2), 217–234 (2007). https://doi.org/10.1016/j.image.2006.11.013

    Article  Google Scholar 

  8. Gopalakrishna, S., Hannuksela, M., Gabbouj, M.: Flexible coding order for 3D video extension of H.265/HEVC. In: Picture Coding Symposium (PCS), pp. 253–256 (2013). https://doi.org/10.1109/PCS.2013.6737731

  9. Müller, K., Vetro, A.: Common test conditions of 3DV core experiments. Doc. JCT3V-G1100, San José (2014)

  10. Afonso, V., Saldanha, M., Conceição, R., Perleberg, M., Porto, M., Zatt, B., Susin, A., Agostini, L.: Real-time architectures for 3D video coding. In: VLSI Architectures for Future Video Coding, ch. 6, pp. 191–226. The Institution of Engineering and Technology (IET), England (2019). https://doi.org/10.1049/PBCS053E_ch6

  11. Lung, C.Y., Shen, C.A.: Design and implementation of a highly efficient fractional motion estimation for the HEVC encoder. J. Real-Time Image Proc. 16, 1541–1557 (2019). https://doi.org/10.1007/s11554-016-0663-2

    Article  Google Scholar 

  12. Alcocer, E., Gutierrez, R., Lopez-Granado, O., Malumbres, M.: Design and implementation of an efficient hardware integer motion estimation for an HEVC video encoder. J. Real-Time Image Proc. 16, 547–557 (2019). https://doi.org/10.1007/s11554-016-0572-4

    Article  Google Scholar 

  13. Shi, L., Gao, X., Yang, X., Chen, Z., Zheng, M.: Algorithm optimization and hardware implementation for merge mode in HEVC. J. Real-Time Image Proc. 17, 623–630 (2020). https://doi.org/10.1007/s11554-018-0818-4

    Article  Google Scholar 

  14. Huang, X., Jia, H., Cai, B., Zhu, C., Liu, J., Yang, M., Xie, D., Gao, W.: Fast algorithms and VLSI architecture design for HEVC intra-mode decision. J. Real-Time Image Proc. 12, 285–302 (2016). https://doi.org/10.1007/s11554-015-0549-8

    Article  Google Scholar 

  15. Afonso, V., Susin, A., Audibert, L., Saldanha, M., Conceição, R., Porto, M., Zatt, B., Agostini, L.: Low-power and high-throughput hardware design for the 3D-HEVC depth intra skip. In: IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–4 (2017). https://doi.org/10.1109/ISCAS.2017.8050463

  16. He, G., Zhou, D., Li, Y., Chen, Z., Zhang, T., Goto, S.: High-throughput power-efficient VLSI architecture of fractional motion estimation for ultra-HD HEVC video encoding. IEEE Trans. Very Large Scale Integr. (VLSI) Syst. 23(12), 3138–3142 (2015). https://doi.org/10.1109/TVLSI.2014.2386897

    Article  Google Scholar 

  17. Jou, S., Chang, S., Chang, T.: Fast motion estimation algorithm and design for real time QFHD high efficiency video coding. IEEE Trans. Circuits Syst. Video Technol. 25(9), 1533–1544 (2015). https://doi.org/10.1109/TCSVT.2015.2389472

    Article  Google Scholar 

  18. Shi, L., Yan, D., Hong, X., Huang, B., Yang, X.: Algorithm optimisation and hardware implementation of interprediction mode decision. J. Real-Time Image Proc. 18, 593–601 (2021). https://doi.org/10.1007/s11554-020-00985-w

    Article  Google Scholar 

  19. Gogoi, S., Peesapati, R.: A hybrid hardware oriented motion estimation algorithm for HEVC/H.265. J. Real-Time Image Proc. 18, 953–966 (2021). https://doi.org/10.1007/s11554-020-01056-w

    Article  Google Scholar 

  20. Afonso, V., Conceição, R., Saldanha, M., Braatz, L., Perleberg, M., Corrêa, G., Porto, M., Agostini, L., Zatt, B., Susin, A.: Energy-aware motion and disparity estimation system for 3D-HEVC with run-time adaptive memory hierarchy. IEEE Trans. Circuits Syst. Video Technol. 29(6), 1878–1892 (2019). https://doi.org/10.1109/TCSVT.2018.2847633

    Article  Google Scholar 

  21. Zhang, Y., Lu, C.: Efficient algorithm adaptations and fully parallel hardware architecture of H.265/HEVC intra encoder. IEEE Trans. Circuits Syst. Video Technol. 29(11), 3415–3429 (2019). https://doi.org/10.1109/TCSVT.2018.2878399

    Article  Google Scholar 

  22. Sanchez, G., Marcon, C., Agostini, L.: Real-time scalable hardware architecture for 3D-HEVC bipartition modes. J. Real-Time Image Proc. 13, 71–83 (2017). https://doi.org/10.1007/s11554-016-0609-8

    Article  Google Scholar 

  23. Amish, F., Bourennane, E.: An efficient hardware solution for 3D-HEVC intra-prediction. J. Real-Time Image Proc. 16, 1559–1571 (2019). https://doi.org/10.1007/s11554-016-0664-1

    Article  Google Scholar 

  24. Ucker, M., Afonso, V., Saldanha, M., Audibert, L., Conceição, R., Susin, A., Porto, M., Zatt, B., Agostini, L.: High-throughput hardware for 3D-HEVC depth-map intra prediction. IEEE Design Test 37(3), 7–14 (2020). https://doi.org/10.1109/MDAT.2019.2952344

    Article  Google Scholar 

  25. Lainema J., Han, W-J.: Intra-picture prediction in HEVC. In: High Efficiency Video Coding (HEVC): Algorithms and Architectures, ch. 4, pp. 91–112. Springer International Publishing (2014). https://doi.org/10.1007/978-3-319-06895-4_4

  26. Zhang, X., Zhang, K., An, J., Huang, H., Lin, J-L., Lei, S.: On lookup table size reduction for DMM1. Doc. JCT3V-J0035, Strasbourg (2014)

  27. Ikai, T., Tsukuba, T.: Simplification of DMM table derivation. Doc. JCT3V-K0042, Geneva (2015)

  28. Lee, J., Park, M., Kim, C.: 3D-CE1: Depth intra skip (DIS) mode. Doc. JCT3V-K0033, Geneva (2015)

  29. Lee, J., Park, M., Choi, B., Cho, Y., Kim, C.: 3D-CE5 related: Segment-wise depth intra mode coding. Doc. JCT3V-G0102, San José (2014)

  30. Zhao, L., Zhang, L., Ma, S., Zhao, D.: Fast mode decision algorithm for intra prediction in HEVC. In: Visual Communications and Image Processing (VCIP), pp. 1–4 (2011). https://doi.org/10.1109/VCIP.2011.6115979

  31. Bjontegaard, G.: Calculation of average PSNR differences between RD curves. Doc. VCEG-M33, Austin (2001)

  32. Cadence. Encounter RTL Compiler [Online]. https://www.cadence.com/content/cadence-www/global/en_US/home/training/all-courses/84441.html

  33. TSMC. 40 nm Technology [Online]. https://www.tsmc.com/english/dedicatedFoundry/technology/logic/l_40nm

  34. Perleberg, M., Goebel, J., Melo, M., Agostini, L., Zatt, B., Porto, M.: ASIC power-estimation accuracy evaluation: a case study using video-coding architectures. In: IEEE 9th Latin American Symposium on Circuits & Systems (LASCAS) (2018). https://doi.org/10.1109/LASCAS.2018.8399919

Download references

Funding

This research was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brasil (CAPES)—Finance Code 001. It was also financed in part by the Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul—Brasil (FAPERGS), and by the Conselho Nacional de Desenvolvimento Científico e Tecnológico—Brasil (CNPq).

Author information

Authors and Affiliations

  1. Video Technology Research Group (ViTech), Pelotas, RS, Brazil

    Murilo R. Perleberg, Vladimir Afonso, Vinicius A. Borges, Bruno Zatt, Luciano V. Agostini & Marcelo Porto

  2. Federal University of Pelotas (UFPel), Pelotas, RS, Brazil

    Murilo R. Perleberg, Vinicius A. Borges, Bruno Zatt, Luciano V. Agostini & Marcelo Porto

  3. Sul-Rio-Grandense Federal Institute (IFSul), Pelotas, RS, Brazil

    Vladimir Afonso

Authors
  1. Murilo R. Perleberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vladimir Afonso
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vinicius A. Borges
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bruno Zatt
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Luciano V. Agostini
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Marcelo Porto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Murilo R. Perleberg.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Perleberg, M.R., Afonso, V., Borges, V.A. et al. Quality-power configurable flexible coding order hardware design for real-time 3D-HEVC intra-frame prediction. J Real-Time Image Proc 19, 969–984 (2022). https://doi.org/10.1007/s11554-022-01238-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-022-01238-8

Keywords

Search

Navigation