Skip to main content

Advertisement

SpringerLink
Log in

HEVC optimization based on human perception for real-time environments

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

High-Efficiency Video Coding (HEVC) is the new emerging video coding standard of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG). The HEVC standard provides a significant improvement in compression efficiency in comparison with existing standards such as H264/AVC by means of greater complexity. In this paper we will examine several HEVC optimizations based on image analysis to reduce its huge CPU, resource and memory expensive encoding process. The proposed algorithms optimize the HEVC quad-tree partitioning procedure, intra/inter prediction and mode decision by means of H264-based methods and spatial and temporal homogeneity analysis which is directly applied to the original video. The validation process of these approaches was conducted by taking into account the human visual system (HVS). The adopted solution makes it possible to perform HEVC real time encoding for HD sequences on a low cost processor with negligible quality loss. Moreover, the frames pre-processing leverages the logic units and embedded hardware available on an Intel GPU, so the execution time of these stages are negligible for the encoding processor.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Ahn S, Lee B, Kim M (2015) A novel fast CU encoding scheme based on spatio-temporal encoding parameters for HEVC inter coding. IEEE Trans Circuits Syst Video Technol 25(3):422–435

    Article  Google Scholar 

  2. Alcocer E, Gutierrez R, Lopez-Granado O, Malumbres MP (2016) Design and implementation of an efficient hardware integer motion estimator for an HEVC video encoder. Journal of Real-Time Image Processing

  3. Bjontegarrd G (2001) Calculation of average PSNR differences between RD curves. ITU-T SC16/Q6 13th VCEG meeting, Austin

    Google Scholar 

  4. Bossen F (2012) Common test conditions and software reference configurations. JCT-VC Document, JCTVC-K1100

  5. Cho S, Kim M (2013) Fast CU splitting and pruning for suboptimal CU partitioning in HEVC intra coding. IEEE Trans Circuits Syst Video Technol 23(9):1555–1564

    Article  Google Scholar 

  6. de Frutos-López M, Orellana-Quirós D, Pujol-Alcolado JC, de María FD (2010) An improved fast mode decision algorithm for intraprediction in H.264/AVC video coding. Signal Process Image Commun 25(10):709–716

    Article  Google Scholar 

  7. Fernandez DG, Del Barrio AA, Botella G, Garcia C (2016) 4K-based intra and inter prediction techniques for HEVC. Proc. SPIE, Real-Time Image and Video Processing 2016, 98970B

  8. Fernández DG, Del Barrio AA, Botella G, García C (2016) Fast CU size decision based on temporal homogeneity detection. In: Design of Circuits and Integrated Systems (DCIS), 2016 Conference on, 1–6

  9. Fernández DG, Del Barrio AA, Botella G, García C (2018) Fast and effective CU size decision based on spatial and temporal homogeneity detection. Multimedia Tools and Applications 77(5):5907–5927

    Article  Google Scholar 

  10. Fernández DG, Del Barrio AA, Botella G, García C, Meyer-Baese U, Meyer-Baese A (2016) HEVC optimizations for medical environments. In: Proc. SPIE 9871, Sensing and Analysis Technologies for Biomedical and Cognitive Applications 2016, 98710B

  11. Fernández DG, Del Barrio AA, Botella G, García C, Prieto M, Hermida R (2018) Complexity reduction in the HEVC/H265 standard based on smooth region classification. Digital Signal Processing 73:24–39

    Article  Google Scholar 

  12. D. G. Fernández, A. A. Del Barrio, Guillermo Botella, Uwe Meyer-Baese, Anke Meyer-Baese, Christos Grecos (2017) Information fusion based techniques for HEVC. Proc. SPIE 10223, Real-Time Image and Video Processing 2017, 102230M. 10.1117/12.2262604

  13. Fraunhofer Institute for Telecommunications (2018) Perceptually optimized video coding. Retrieved from: https://www.hhi.fraunhofer.de/en/departments/vca/research-groups/image-video-coding/research-topics/perceptually-optimized-video-coding.html. Accessed 17 Dec 2018

  14. Goswami K, Lee J-H, Jang K-S, Kim B-G, Kwon K-K (2014) Entropy difference-based early skip detection technique for high efficiency video coding. Journal of Real-Time Image Processing

  15. He G, Zhou D, Goto S (2013) Transform-based fast mode and depth decision algorithm for HEVC intra prediction. IEEE 10th International Conference on ASIC (ASICON), pp. 1–4

  16. Intel Corporation (2016) Introduction to advance motion extension for OpenCL. Retrieved from: https://software.intel.com/en-us/articles/intro-to-advanced-motion-estimation-extension-for-opencl. Accessed 17 Dec 2018

  17. Intel Corporation White Paper (2012) Performance Interactions of OpenCL* Code and Intel® Quick Sync Video on Intel® HD Graphics 4000

  18. ITU-R BT.500 (2012) Methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union, Geneva

    Google Scholar 

  19. Jiang W, Ma H, Chen Y (2012) Gradient based fast mode decision algorithm for intra prediction in HEVC. In: 2nd International Conference on Consumer Electronics, Communications and Networks (CECNet), pp. 1836–1840

  20. Jiang W, Ma H, Chen Y (2012) Gradient Based Fast Mode Decision Algorithm for Intra Prediction in HEVC. In: International Conference on Consumer Electronics, Communications and Networks (CECNet)

  21. Khan MUK, Shafique M, Grellert M, Henkel J (2013) Hardware-software collaborative complexity reduction scheme for the emerging HEVC intra encoder. Design, Automation Test in Europe Conference Exhibition (DATE) 2013:125–128

    Article  Google Scholar 

  22. Khronos OpenCL Working Group (2011) The OpenCL specification version 1.1. Revision 44

  23. Khronos OpenCL Working Group (2016) Online documentation for cl_intel_advaanced_motion_estimation. Retrieved from: https://www.khronos.org/registry/cl/extensions/intel/cl_intel_advanced_motion_estimation.txt. Accessed 17 Dec 2018

  24. Koumaras H, Kourtis M, Martakos D (2012) Benchmarking the encoding efficiency of h.265/HEVC and h.264/AVC,” in Future Network Mobile Summit (FutureNetw)

  25. Leal da Silva T, Agostini LV, da Silva Cruz LA (2015) Fast intra prediction algorithm based on texture analysis for 3D-HEVC encoders” in Journal of Real-Time Image Processing

  26. Lee JH, Park CS, Kim BG, Jun DS, Jung SH, Choi JS (2013) Novel fast PU decision algorithm for the HEVC video standard”, IEEE International Conference on Image Processing (ICIP)

  27. Li C, Bovik AC (2009) Three-component weighted structural similarity index. Proc. SPIE 7242–72420, Image Quality and System Performance VI

  28. Lim K, Lee J, Kim S, Lee S (2015) Fast PU skip and split termination algorithm for HEVC intra. IEEE Transactions on Circuits Systems for Video Technology 25(8)

  29. Liu X, Liu Y, Wang P, Lai C, Chao H (2016) An adaptive mode Decision algorithm based on video texture characteristics for HEVC intra prediction. IEEE Transactions on Circuits Systems for Video Technology, vol. PP: 99

  30. Mallikarachchi T, Fernando A, Arachchi H (2014) Efficient coding unit size selection based on texture analysis for HEVC intra prediction. IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6

  31. McCann K, Rosewarne C, Bross B, Naccari M, Sharman K, Sullivan G (2014) High efficiency video coding (HEVC) encoder description 0v16 (HM16). JCT-VC High Efficiency Video Coding N14 703

  32. Min B, Cheung R (2014) A fast cu size decision algorithm for HEVC intra encoder. IEEE Transactions on Circuits and Systems for Video Technology PP(99):1

    Google Scholar 

  33. Moorthy AK, Bovik AC (2010) Efficient motion weighted spatio-temporal video SSIM Index. Proc. SPIE 7527–75271, Human Vision and Electronic Imaging XV

  34. MSU Graphics & Media Lab (Video Group) (2016) MSU video quality measurement tool. Retrieved from: http://www.compression.ru/video/quality_measure/video_measurement_tool.html. Accessed 17 Dec 2018

  35. Multimedia Signal Processing Group (MMSPG) (2016) VQMT: video quality measurement tool. Retrieved from: http://mmspg.epfl.ch/vqmt. Accessed 17 Dec 2018

  36. Na S, Lee W, Yoo K (2014) Edge-based fast mode decision algorithm for intra prediction in HEVC. IEEE International Conference on Consumer Electronics (ICCE), pp. 11–14

  37. Öztekin A, ErÇelebi E (2015) An early split and skip algorithm for fast intra CU selection in HEVC. In: Journal of Real-Time Image Processing

  38. Pastuszak G, Trochimiuk M (2015) Algorithm and architecture design of the motion estimation for the H.265/HEVC 4K-UHD encoder. Journal of Real-Time Image Processing

  39. Ponomarenko N, Silvestri F, Egiazarian K, Carli M, Astola J, Lukin V (2007) On Between-Coefficient Contrast Masking of DCT Basis Functions. Third International Workshop on Video Processing and Quality Metrics (VPQM), Scottsdale

    Google Scholar 

  40. Ramezanpour M, Zargari F (2016) Fast HEVC I-frame coding based on strength of dominant direction of CUs. Journal of Real-Time Image Processing

  41. Shang X, Wang G, Fan T, Li Y (2015) Fast CU size decision and PU mode decision algorithm in HEVC intra coding. IEEE International Conference on Image Processing (ICIP)

  42. Sheikh HR, Bovik AC (2006) Image Information and Visual Quality. IEEE Trans Image Process 15(2):430–444

    Article  Google Scholar 

  43. Sheikh HR, Bovik AC, de Veciana G (2005) An Information Fidelity Criterion for Image Quality Assessment Using Natural Scene Statistics. IEEE Trans Image Process 14(12):2117–2128

    Article  Google Scholar 

  44. Shen L, Liu Z, Liu S, Zhang Z, An P (2009) Selective disparity estimation and variable size motion estimation based on motion homogeneity for multi-view coding. IEEE Transactions on Broadcasting 55(4)

  45. Shen L, Liu Z, Yan T, Zhang Z, An P (2010) View-adaptive motion estimation and disparity estimation for low complexity multiview video coding. IEEE Transactions on Circuits and Systems for Video Technology 20(6)

  46. Shen L, Liu Z, Zhang Z, Shi X (2008) Fast Inter Mode Decision Using Spatial Property of Motion Field. IEEE Transactions on Multimedia 10(6):1208–1214

    Article  Google Scholar 

  47. Shen L, Liu Z, Zhang X, Zhao W, Zhang Z (2013) An effective CU size decision method for HEVC encoders. IEEE Transactions on Multimedia 15(2):465–470

    Article  Google Scholar 

  48. Shen L, Zhang Z, An P (2013) Fast CU size decision and mode decision algorithm for HEVC intra coding. IEEE Transactions on Consumer Electronics 59(1)

  49. Shen L, Zhang Z, Liu Z (2014) Effective CU size decision for HEVC intracoding. IEEE Trans Image Process 23(10):4232–4241

    Article  MathSciNet  Google Scholar 

  50. Sullivan G, Ohm J, Han W-J, Wiegand T (2012) Overview of the high efficiency video coding (hevc) standard. IEEE Trans Circuits Syst Video Technol 22(12):1649–1668

    Article  Google Scholar 

  51. Sze V, Budagavi M, Sullivan GJ (2014) High Efficiency Video Coding (HEVC), 1st ed. Springer International Publishing. Available: https://link.springer.com/book/10.1007%2F978-3-319-06895-4. Accessed 17 Dec 2018

  52. Tian G, Goto S (2012) Content adaptive prediction unit size decision algorithm for HEVC intra coding,” in Picture Coding Symposium (PCS), pp. 405–408

  53. Ting Y-C, Chang T-S (2014) Gradient-based PU size selection for HEVC intra prediction. IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1929–1932

  54. Vanne J, Viitanen M, Hämäläinen TD (2014) Efficient mode decision schemes for HEVC inter prediction. IEEE Transactions on Circuits and Systems for Video Technology 24(9):1579–1593

    Article  Google Scholar 

  55. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  56. Wang H-M, Lin J-K, Yang J-F (2006) Fast inter mode decision based on hierarchical homogeneous detection and cost analysis for h.264/AVC coders. In: IEEE International Conference on Multimedia and Expo, pp. 709–712

  57. Wang Z, Simoncelli EP, Bovik AC (2003) Multi-scale structural similarity for image quality assessment. In: Proceedings of the 37th IEEE Asiloma Conference on Signal, Systems and Computers, Pacific Grove

  58. Watson AB (1998) Toward a perceptual video quality metric. Human Vision, Visual Processing, and Digital Display VIII(3299):139–147

    Google Scholar 

  59. Wu D, Pan F, Lim K, Wu S, Li Z, Lin X, Rahardja S, Ko C (2005) Fast intermode decision in h.264/AVC video coding. IEEE Transactions on Circuits and Systems for Video Technology 15(7):953–958

    Article  Google Scholar 

  60. x265 project (2017) http://x265.org/. GNU GPL 2 license, source code available at: https://bitbucket.org/multicoreware/x265/wiki/Home. Accessed 17 Dec 2018

  61. Xiao F (2000) DCT-based Video Quality Evaluation---Final Project for EE392J

  62. Xiong J, Li H, Meng F, Wu Q, Ngan KN (2015) Fast HEVC inter CU decision based on latent SAD estimation. IEEE Transactions on Multimedia 17(12):2147–2159

    Article  Google Scholar 

  63. Xiong J, Li H, Meng F, Zhu S, Wu Q, Zeng B (2014) MRF-based fast HEVC inter CU decision with the variance of absolute differences. IEEE Trans Multimedia 16(8):2141–2153

    Article  Google Scholar 

  64. Xiong J, Li H, Wu Q, Meng F (2014) A fast HEVC inter CU selection method based on pyramid motion divergence. IEEE Transactions on Multimedia 16(2):559–564

    Article  Google Scholar 

  65. xiph.org (2016) Derf’s test media collection. Retrieved from: https://media.xiph.org/video/derf/. Accessed 17 Dec 2018

  66. Ye T, Zhang D, Dai F, Zhang Y (2013) Fast mode decision algorithm for intra prediction in HEVC. in Proceedings of the Fifth International Conference on Internet Multimedia Computing and Service, pp. 300–304

  67. Ye T, Zhang D, Dai F, Zhang Y (2013) Fast mode decision algorithm for intra prediction in HEVC. in the Fifth International Conference on Internet Multimedia Computing and Service (ICIMCS'13)

  68. Zhang Y, Li Z, Li B (2012) Gradient-based fast decision for intra prediction in HEVC. In Visual Communications and Image Processing (VCIP), pp. 1–6

  69. Zhang H, Ma Z (2014) Fast Intra mode decision for high efficiency video coding (HEVC). IEEE Transactions on Circuits Systems for Video Technology 24(4):660–668

    Article  Google Scholar 

  70. Zhang H, Ma Z (2016) Fast intra mode and CU size decision for HEVC. IEEE Transactions on Circuits Systems for Video Technology PP(99):1–7

    Google Scholar 

Download references

Acknowledgments

This work has been partially supported by Spanish research projects TIN 2015-65277-R and TIN-2012-32180, as well as the UCM-Banco Santander Grant PR26-16/20B-1.

Author information

Authors and Affiliations

  1. Department of Computer Architecture and Automation, Complutense University of Madrid, Ciudad Universitaria s/n, 28040, Madrid, Spain

    D. G. Fernández, Guillermo Botella, Alberto A. Del Barrio, Carlos García & Manuel Prieto-Matías

  2. Department of Computer Science, Central Washington University, Washington, WA, 98926-7520, USA

    Christos Grecos

Authors
  1. D. G. Fernández
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guillermo Botella
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alberto A. Del Barrio
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Carlos García
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Manuel Prieto-Matías
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Christos Grecos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alberto A. Del Barrio.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fernández, D.G., Botella, G., Del Barrio, A.A. et al. HEVC optimization based on human perception for real-time environments. Multimed Tools Appl 79, 16001–16033 (2020). https://doi.org/10.1007/s11042-018-7033-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-7033-y

Keywords

Search

Navigation