Skip to main content
SpringerLink
Log in

Proof-of-concept: role of generic content characteristics in optimizing video encoders

Complexity- and content-aware sequence-level encoder parameter decision framework

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The influence of content characteristics on the efficiency of redundancy and irrelevance reduction in video coding is well known. Each new standard in video coding includes additional coding tools that potentially increase the complexity of the encoding process in order to gain further rate-distortion efficiency. In order to be versatile, encoder implementations often neglect the content dependency or they optimize the encoding complexity on a local scale, i.e. on a single frame or on the coding unit level without being aware of the global content type. In this contribution, an analysis is presented which coding tool settings of the recent High Efficiency Video Coding (HEVC) standard are most efficient for a given content type when balancing rate-distortion against computational complexity measured in encoding time. The content type is algorithmically determined, leading to a framework for rate-distortion-complexity based encoder parameter decision for any given video sequence. The implementability is demonstrated using a set of 35 Ultra-HD (UHD) sequences. The performance results and evaluations show that the encoding parameters may be predicted to optimize the video coding. For instance, predicting motion search range achieves complexity reduction of 36% on average when HEVC reference HM is used at a cost of bitrate (2%). When another HEVC coding standard software, x265, is used to predict the coding unit (CU) size, there is a reduction of 20% in bitrate and of 8% in distortion but there is a reduction of 6% in execution time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. Group UV (2013 (accessed May 15, 2014)) Ultra Video Group 4K sequences. http://ultravideo.cs.tut.fi/#testsequences

  2. Xiph.org Video Test Media 4K sequences (accessed May 15, 2014). https://media.xiph.org/video/derf/

  3. HEVC software reference (accessed May 2, 2014). http://hevc.hhi.fraunhofer.de/

  4. x265 software reference (accessed May 2, 2014). https://bitbucket.org/multicoreware/x265/wiki/Home

  5. Orange software (accessed July 2, 2014). http://orange.biolab.si/

References

  1. Agrafiotis D, Bull DR, Canagarajah CN (2006) Enhanced error concealment with mode selection. IEEE Trans Circuits Syst Video Technol 16(8):960–973

    Article  Google Scholar 

  2. Aldahdooh A, Barkowsky M, Le Callet P (2015) The impact of complexity in the rate-distortion optimization: a visualization tool. In: Systems, signals and image processing (IWSSIP), 2015 international conference. IEEE, pp 45–48

  3. Aldahdooh A, Barkowsky M, Callet PL (2016) Content-aware adaptive multiple description coding scheme. In: 2016 IEEE international conference on multimedia expo workshops(ICMEW), 22nd international packet video workshop, pp 1–6

  4. Aldahdooh A, Masala E, Janssens O, Wallendael GV, Barkowsky M (2016) Comparing simple video quality measures for loss-impaired video sequences on a large-scale database. In: 2016 Eighth international conference on quality of multimedia experience (QoMEX). https://doi.org/10.1109/QoMEX.2016.7498941, pp 1–6

  5. Bossen F, Bross B, Suhring K, Flynn D (2012) HEVC Complexity and implementation analysis. IEEE Trans Circuits Syst Video Technol 22(12):1685–1696

    Article  Google Scholar 

  6. Cen S, Cosman PC (2003) Decision trees for error concealment in video decoding. IEEE Trans Multimedia 5(1):1–7

    Article  Google Scholar 

  7. Cen YF, Wang WL, Yao XW (2015) A fast cu depth decision mechanism for HEVC. Inf Process Lett 115(9):719–724

    Article  MathSciNet  MATH  Google Scholar 

  8. Chiu MY, Siu WC (2010) Computationally-scalable motion estimation algorithm for h.264/AVC video coding. IEEE Trans Consum Electron 56(2):895–903

    Article  Google Scholar 

  9. Chen H, Xie R, Zhang L (2015) Gradient based fast mode and depth decision for high efficiency intra frame video coding. In: Broadband multimedia systems and broadcasting (BMSB), 2015 IEEE International Symposium. IEEE, pp 1–6

  10. Corrêa G, Assuncao P, da Silva Cruz LA, Agostini L (2012) Adaptive coding tree for complexity control of high efficiency video encoders. In: Picture coding symposium (PCS), 2012. IEEE, pp 425–428

  11. Correa G, Assuncao PA, Volcan Agostini L, da Silva Cruz LA (2015) Fast HEVC encoding decisions using data mining. IEEE Trans Circuits Syst Video Technol 25(4):660–673

    Article  Google Scholar 

  12. Egiazarian K, Astola J, Ponomarenko N, Lukin V, Battisti F, Carli M (2006) New full-reference quality metrics based on HVS. In: CD-ROM proceedings of the second international workshop on video processing and quality metrics, Scottsdale, USA, vol 4

  13. Eickeler S, Muller S (1999) Content-based video indexing of tv broadcast news using hidden markov models. In: Proceedings of the 1999 IEEE international conference on acoustics, speech, and signal processing, 1999, vol 6. IEEE, pp 2997–3000

  14. Group TVQE (2008) The validation of objective models of multimedia quality assessment

  15. Hampapur A, Jain R, Weymouth T (1994) Digital video indexing in multimedia systems. University of Michigan, Artificial Intelligence Laboratory

  16. Haralick RM, Shanmugam K, Dinstein IH (1973) Textural features for image classification. IEEE Trans Syst, Man Cybern SMC-3(6):610–621

    Article  Google Scholar 

  17. He J, Yang W, Wang J (2015) Fast HEVC coding unit decision based on bp-neural network. Int J Grid Distributed Comput 8(4):289–300

    Article  Google Scholar 

  18. Huang SC, Kuo SY (2008) Optimization of hybridized error concealment for h. 264. IEEE Trans Broadcast 54(3):499–516

    Article  Google Scholar 

  19. ITU (2013) ITU-T H.265: High efficiency video coding. http://www.itu.int/rec/T-REC-H.265-201504-I

  20. ITU-T RECOMMENDATION P (1999) Subjective video quality assessment methods for multimedia applications

  21. Kim JH, Kim BG (2008) Fast block mode decision algorithm in h. 264/AVC video coding. J Vis Commun Image Represent 19(3):175–183

    Article  Google Scholar 

  22. Korhonen J, You J (2010) Improving objective video quality assessment with content analysis. In: Proceedings of the Fifth international workshop on video processing and quality metrics for consumer electronics (VPQM) Scottsdale, USA

  23. La B, Eom M, Choe Y (2007) Fast mode decision for intra prediction in H. 264/AVC encoder. In: Image Processing, 2007. ICIP 2007. IEEE International Conference. IEEE, vol 5, pp V–321

  24. Leng J, Sun L, Ikenaga T, Sakaida S (2011) Content based hierarchical fast coding unit decision algorithm for HEVC. In: 2011 International Conference on Multimedia and Signal Processing (CMSP). IEEE, vol 1, pp 56–59

  25. Lewis J (1995) Fast normalized cross-correlation. In: Vision interface, vol 10, pp 120–123

  26. Li X, Wien M, Ohm JR (2011) Rate-complexity-distortion optimization for hybrid video coding. IEEE Trans Circuits Syst Video Technol 21(7):957–970

    Article  Google Scholar 

  27. Meng B, Au OC, Wong CW, Lam HK (2003) Efficient intra-prediction mode selection for 4 ×4 blocks in H. 264 Multimedia and Expo, 2003. ICME ’03. In: Proceedings. 2003 international conference on, 2003. IEEE, vol 3, pp III–521

  28. Nguyen VA, Do MN (2015) Efficient coding unit size selection for HEVC downsizing transcoding. In: Circuits and Systems (ISCAS), 2015 IEEE International Symposium. IEEE, pp 1286–1289

  29. Ortiz-Jaramillo B, Niño-Castañeda J, Platiša L, Philips W (2016) Content-aware objective video quality assessment. J Electron Imaging 25 (1):011,013–013,011

    Article  Google Scholar 

  30. Pinson MH, Wolf S (2004) A new standardized method for objectively measuring video quality. IEEE Trans Broadcast 50(3):312–322

    Article  Google Scholar 

  31. Pinson MH, Barkowsky M, Le Callet P (2013) Selecting scenes for 2d and 3d subjective video quality tests. EURASIP J Image Video Process 2013(1):1–12

    Article  Google Scholar 

  32. Pinson MH, Sue Boyd K, Hooker J, Muntean K (2013) How to choose video sequences for video quality assessment. In: Proceedings of the seventh international workshop on video processing and quality metrics for consumer electronics (VPQM) Scottsdale, USA

  33. Pitrey Y, Barkowsky M, Pépion R, Le Callet P, Hlavacs H (2012) Influence of the source content and encoding configuration on the perceived quality for scalable video coding. In: IS&T/SPIE electronic imaging, international society for optics and photonics, pp 82,911K–82,911K

  34. Ponomarenko N, Silvestri F, Egiazarian K, Carli M, Astola J, Lukin V (2007) On between-coefficient contrast masking of DCT basis functions. In: Proceedings of the third international workshop on video processing and quality metrics, vol 4

  35. Rongfu Z, Yuanhua Z, Xiaodong H (2004) Content-adaptive spatial error concealment for video communication. IEEE Trans Consum Electron 50(1):335–341

    Article  Google Scholar 

  36. Saponara S, Casula M, Rovati F, Alfonso D, Fanucci L (2006) Dynamic control of motion estimation search parameters for low complex H. 264 video coding. IEEE Transactions on Consumer Electronics 52(1):232–239

    Article  Google Scholar 

  37. Sheikh HR, Bovik AC (2006) Image information and visual quality. IEEE Trans Image Process 15(2):430–444

    Article  Google Scholar 

  38. Shen X, Yu L (2013) CU Splitting early termination based on weighted SVM. EURASIP J Image Video Process 2013(1):1–11

    Article  Google Scholar 

  39. Shen L, Zhang Z, Liu Z (2014) Adaptive inter-mode decision for HEVC jointly utilizing inter-level and spatiotemporal correlations. IEEE Trans Circuits Syst Video Technol 24(10):1709–1722

    Article  Google Scholar 

  40. Shen L, Zhang Z, Liu Z (2014) Effective CU size decision for HEVC intracoding. IEEE Trans Image Process 23(10):4232–4241

    Article  MathSciNet  MATH  Google Scholar 

  41. Sobel I, Feldman G (1968) A 3 × 3 isotropic gradient operator for image processing. a talk at the Stanford Artificial Project in pp 271–272

  42. Song L, Tang X, Zhang W, Yang X, Xia P (2013) The SJTU 4K video sequence dataset. In: QoMEX, pp 34–35

  43. Srinivasan G, Shobha G (2008) Statistical texture analysis. In: Proceedings of world academy of science, engineering and technology, vol 36, pp 1264–1269

  44. Su L, Lu Y, Wu F, Li S, Gao W (2009) Complexity-constrained h. 264 video encoding. IEEE Trans Circuits Syst Video Technol 19(4):477–490

    Article  Google Scholar 

  45. Sullivan GJ, Wiegand T (1998) Rate-distortion optimization for video compression. IEEE Signal Process Mag 15(6):74–90

    Article  Google Scholar 

  46. Sullivan GJ, Ohm J, Han WJ, Wiegand T (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Trans Circuits Syst Video Technol 22 (12):1649–1668

    Article  Google Scholar 

  47. Systems C (2017) The zettabyte era: trends and analysis

  48. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13 (4):600–612

    Article  Google Scholar 

  49. Wang J, Dong L, Xu Y (2015) A fast inter prediction algorithm based on rate-distortion cost in HEVC. Int J Signal Process Image Process Pattern Recogn 8 (11):141–158

    Google Scholar 

  50. Wang Z, Simoncelli EP, Bovik AC (2003) Multiscale structural similarity for image quality assessment. In: Conference Record of the Thirty-Seventh Asilomar Conference. IEEE, vol 2, pp 1398–1402

  51. Wolf S, Pinson M (2002) Video quality measurement techniques. National telecommunications and information administration (NTIA) Report

  52. Yu J, Srinath M (2001) An efficient method for scene cut detection. Pattern Recogn Lett 22(13):1379–1391

    Article  MATH  Google Scholar 

  53. Zhang L, Gao W (2007) Reusable architecture and complexity-controllable algorithm for the integer/fractional motion estimation of h. 264. IEEE Trans Consum Electron 53(2):749–756

    Article  Google Scholar 

  54. Zhao D, Zhu S, Gao S (2015) A novel fast intra-prediction algorithm for high-efficiency video coding based on structural similarity. Optik-Int J Light Electron Optics 126(23):4212–4218

    Article  Google Scholar 

  55. Zhu K, Asari V, Saupe D (2013) No-reference quality assessment of H.264/AVC encoded video based on natural scene features. In: SPIE defense, security, and sensing, international society for optics and photonics, pp 875,505–875,505

  56. Zhu K, Hirakawa K, Asari V, Saupe D (2013) A no-reference video quality assessment based on laplacian pyramids. In: ICIP, pp 49–53

Download references

Acknowledgements

This work is supported by the Marie Sktodowska-Curie under the PROVISION (PeRceptually Optimised VIdeo CompresSION) project bearing Grant Number 608231 and Call Identifier: FP7-PEOPLE-2013-ITN.

Author information

Authors and Affiliations

  1. Laboratoire des Sciences du Numérique de Nantes (LS2N) – UMR 6004, Université de Nantes, Nantes, France

    Ahmed Aldahdooh, Marcus Barkowsky & Patrick Le Callet

Authors
  1. Ahmed Aldahdooh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcus Barkowsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patrick Le Callet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmed Aldahdooh.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aldahdooh, A., Barkowsky, M. & Le Callet, P. Proof-of-concept: role of generic content characteristics in optimizing video encoders. Multimed Tools Appl 77, 16069–16097 (2018). https://doi.org/10.1007/s11042-017-5180-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5180-1

Keywords

Search

Navigation