Skip to main content
SpringerLink
Log in

An overview of rate control techniques in HEVC and SHVC video encoding

  • 1174: Futuristic Trends and Innovations in Multimedia Systems Using Big Data, IoT and Cloud Technologies (FTIMS)
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Video standards are crucial for exchanging video content, enabling a myriad of services and supporting a wide variety of devices ranging from personal devices to clouds and IoT. One of the core requirements in video standards is the rate control that regulates the bit allocation and picture quality. This paper presents an overview of rate control techniques in the HEVC video coding standard. While providing an insight into the rate control mechanism specific to HEVC, it describes the basic operating principle of rate control algorithms, including their essential parameter, outputs, and performance measures. We review rate control in past coding standards and bring out the basic features of HEVC that drive the need for new rate control algorithms. Alongside, we delineate the Rate-Distortion model-based taxonomy of various algorithms, including their classification criteria. The paper gives out another classification of the rate control algorithms based on their basic principle and mechanisms. The article also explains the scalable extension of HEVC, namely SHVC, while highlighting some of the possible SHVC rate control design challenges. Finally, we present some of the unresolved research issues in HEVC rate control and outline possible future research directions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Ahmad I, Luo J (2006) On using game theory to optimize the rate control in video coding. IEEE Trans Circ Syst Vid Technol 16(2):209–219. https://doi.org/10.1109/TCSVT.2005.856899

    Article  Google Scholar 

  2. Ahmad I, Wei X, Sun Y, Zhang YQ (2005) Video transcoding: an overview of various techniques and research issues. IEEE Trans Multimed 7(5):793–804. https://doi.org/10.1109/TMM.2005.854472

    Article  Google Scholar 

  3. Altunbasak Y, Kamaci N (2004) An analysis of the DCT coefficient distribution with the H.264 video coder. In: 2004 IEEE International conference on acoustics, speech, and signal processing, vol 3. pp iii–177–80. https://doi.org/10.1109/ICASSP.2004.1326510

  4. Biatek T, Raulet M, Travers JF, Deforges O (2014) Efficient quantization parameter estimation in hevc based on ρ-domain, IEEE

  5. Bossen F, Bross B, Suhring K, Flynn D (2012) Hevc complexity and implementation analysis. IEEE Trans Circ Syst Vid Technol 22 (12):1685–1696

    Article  Google Scholar 

  6. Boyce JM, Ye Y, Chen J, Ramasubramonian AK (2016) Overview of shvc: Scalable extensions of the high efficiency video coding standard. IEEE Trans Circ Syst Vid Technol 26(1):20–34. https://doi.org/10.1109/TCSVT.2015.2461951

    Article  Google Scholar 

  7. Chen Y, Wang B, Liu KR (2009) A game-theoretic framework for multi-user multimedia rate allocation. In: IEEE international conference on acoustics, speech and signal processing, 2009. ICASSP 2009. IEEE, pp 1987–2000

  8. Chen Z, Pan X (2019) An optimized rate control for low-delay h.265/hevc. IEEE Trans Image Process 28(9):4541–4552. https://doi.org/10.1109/TIP.2019.2911180

    Article  MathSciNet  MATH  Google Scholar 

  9. Chiang T, Zhang YQ (1997) A new rate control scheme using quadratic rate distortion model. IEEE Trans Circ Syst Vid Technol 7(1):246–250

    Article  Google Scholar 

  10. Choi H, Nam J, Yoo J, Sim D, Bajic I (2012) Rate control based on unified RQ model for HEVC. ITU-T SG16 Contribution, JCTVC-H0213, pp 1–13

  11. Choi H, Yoo J, Nam J, Sim D, Bajić IV (2013) Pixel-wise unified rate-quantization model for multi-level rate control. IEEE J Selected Topics Signal Process 7(6):1112–1123

    Article  Google Scholar 

  12. Dayananda USM, Swaminathan V (2015) Investigating scalable high efficiency video coding for http streaming. In: 2015 IEEE international conference on multimedia & expo workshops (ICMEW). IEEE, pp 1–6

  13. Guo H, Zhu C, Xu M, Li S (2020) Inter-block dependency-based ctu level rate control for hevc. IEEE Trans Broadcast 66(1):113–126. https://doi.org/10.1109/TBC.2019.2917402

    Article  Google Scholar 

  14. Guo Y, Zou B, Ren J, Liu Q, Zhang D, Zhang Y (2019) Distributed and efficient object detection via interactions among devices, edge, and cloud. IEEE Trans Multimed 21(11):2903–2915. https://doi.org/10.1109/TMM.2019.2912703

    Article  Google Scholar 

  15. Gweon R, Lee YL (2012) N-level quantization in HEVC. In: 2012 IEEE international symposium on broadband multimedia systems and broadcasting (BMSB), IEEE, pp 1–5

  16. Hang HM, Chen JJ (1997) Source model for transform video coder and its application. I. Fundamental theory. IEEE Trans Circ Syst Vid Technol 7 (2):287–298

    Article  Google Scholar 

  17. He Z, Mitra SK (2001) A unified rate-distortion analysis framework for transform coding. IEEE Trans Circ Syst Vid Technol 11(12):1221–1236

    Article  Google Scholar 

  18. Helle P, Lakshman H, Siekmann M, Stegemann J, Hinz T, Schwarz H, Marpe D, Wiegand T (2013) A scalable video coding extension of HEVC. In: Data compression conference (DCC), 2013. IEEE, pp 201–210

  19. He Zhihai, Kim Yong Kwan, Mitra SK (2001) Low-delay rate control for dct video coding via /spl rho/-domain source modeling. IEEE Trans Circ Syst Vid Technol 11(8):928–940. https://doi.org/10.1109/76.937431

    Article  Google Scholar 

  20. Hidaka T, Ozawa K (1993) Iso/iec jtc1 sc29/wg11; report on mpeg-2 subjective assessment at kurihama. Signal Process Image Commun 5 (1):127–157. https://doi.org/10.1016/0923-5965(93)90032-O, http://www.sciencedirect.com/science/article/pii/092359659390032O

    Article  Google Scholar 

  21. Ho YH, Jin GL, Liang Y, Peng WH, Li X (2021) A dual-critic reinforcement learning framework for frame-level bit allocation in HEVC/H.265. In: 2021 Data compression conference (DCC). pp 13–22. https://doi.org/10.1109/DCC50243.2021.00009

  22. Hsu CH, Hefeeda M (2008) Optimal coding of multilayer and multiversion video streams. IEEE Trans Multimed 10(1):121–131

    Article  Google Scholar 

  23. Hu S, Wang H, Kwong S, Zhao T, Kuo CCJ (2011) Rate control optimization for temporal-layer scalable video coding. IEEE Trans Circ Syst Vid Technol 21(8):1152–1162

    Article  Google Scholar 

  24. Hu S, Wang H, Kwong S (2012) Adaptive quantization-parameter clip scheme for smooth quality in H. 264/AVC. IEEE Trans Image Process 21 (4):1911–1919

    Article  MathSciNet  MATH  Google Scholar 

  25. Hussain T, Muhammad K, Ser JD, Baik SW, de Albuquerque VHC (2020) Intelligent embedded vision for summarization of multiview videos in IIoT. IEEE Trans Ind Inf 16(4):2592–2602

    Article  Google Scholar 

  26. Jing X, Chau LP, Siu WC (2008) Frame complexity-based rate-quantization model for H. 264/AVC intraframe rate control. IEEE Signal Process Lett 15:373–376

    Article  Google Scholar 

  27. Kamaci N, Altunbasak Y, Mersereau RM (2005) Frame bit allocation for the H. 264/AVC video coder via cauchy-density-based rate and distortion models. IEEE Trans Circ Syst Vid Technol 15(8):994–1006

    Article  Google Scholar 

  28. Kwok YK, Karlapalem K, Ahmad I, Pun NM (1996) Design and evaluation of data allocation algorithms for distributed multimedia database systems. IEEE J Selected Areas Commun 14(7):1332–1348

    Article  Google Scholar 

  29. Kwon DK, Shen MY, Kuo CCJ (2007) Rate control for H. 264 video with enhanced rate and distortion models. IEEE Trans Circ Syst Vid Technol 17(5):517–529

    Article  Google Scholar 

  30. Lee B, Kim M (2012) A rate control scheme for high efficiency video coding using a new rate-quantization model. In: Proceedings of the international conference on image processing, computer vision, and pattern recognition (IPCV)

  31. Lee B, Kim M, Nguyen TQ (2014) A frame-level rate control scheme based on texture and nontexture rate models for high efficiency video coding. IEEE Trans Circ Syst Vid Technol 24(3):465–479

    Article  Google Scholar 

  32. Li B, Li H, Li L, Zhang J (2012) Rate control by R-lambda model for HEVC. In: JCTVC-K0103, JCTVC of ISO/IEC and ITU-T, 11th meeting Shanghai, China

  33. Li B, Li H, Li L, Zhang J (2014) λ domain rate control algorithm for high efficiency video coding. https://doi.org/10.1109/TIP.2014.2336550, vol 23, pp 3841–3854

  34. Li L, Li H (2015) λ domain based optimal bit allocation for scalable high efficiency video coding. In: 2015 IEEE international symposium on circuits and systems (ISCAS). IEEE, pp 2788–2791

  35. Li L, Li Z, Liu S, Li H (2020) Rate control for video-based point cloud compression. IEEE Trans Image Process 29:6237–6250. https://doi.org/10.1109/TIP.2020.2989576

    Article  MathSciNet  MATH  Google Scholar 

  36. Li S, Xu M, Deng X, Wang Z (2015) Weight-based R-λ rate control for perceptual HEVC coding on conversational videos. Signal Process Image Commun 38:127–140

    Article  Google Scholar 

  37. Lin Y, Yang M, Lin Y (2019) Low-cost four-dimensional experience theater using home appliances. IEEE Trans Multimed 21(5):1161–1168

    Article  MathSciNet  Google Scholar 

  38. Liu Y, Li ZG, Soh YC (2008) Region-of-interest based resource allocation for conversational video communication of H. 264/AVC. IEEE Trans Circ Syst Vid Technol 18(1):134–139

    Article  Google Scholar 

  39. Luo J, Ahmad I, Liang Y, Swaminathan V (2008) Motion estimation for content adaptive video compression. IEEE Trans Circ Syst Vid Technol 18(7):900–909

    Article  Google Scholar 

  40. Luo J, Ahmad I, Sun Y (2010) Controlling the bit rate of multi-object videos with noncooperative game theory. IEEE Trans Multimed 12(2):97–107

    Article  Google Scholar 

  41. Ma S, Gao Wen, Yan L (2005) Rate-distortion analysis for h.264/avc video coding and its application to rate control. IEEE Trans Circ Syst Vid Technol 15(12):1533–1544

    Article  Google Scholar 

  42. Max J (1960) Quantizing for minimum distortion. IRE Trans Inf Theory 6(1):7–12

    Article  MathSciNet  Google Scholar 

  43. Meddeb M, Cagnazzo M, Pesquet-Popescu B (2014) Region-of-interest-based rate control scheme for high-efficiency video coding. APSIPA Trans Signal Inf Process 3

  44. pixeltools (2012) Rate control and H.264. http://www.pixeltools.com/rate_control_paper.html, (accessed May. 25, 2021)

  45. Poynton C (2003) Digital video and HDTV algorithms and interfaces, 1st edn. Morgan Kaufmann Publishers Inc., San Francisco

    Google Scholar 

  46. Ramanand AA, Ahmad I, Swaminathan V (2017) A survey of rate control in hevc and shvc video encoding. In: 2017 IEEE international conference on multimedia expo workshops (ICMEW). pp 145–150

  47. Salah M, El-Shweky B, ElKholy K, Helmy A, Ismail Y, Salah K (2018) Hevc implementation for iot applications. In: 2018 30th international conference on microelectronics (ICM). pp 295–298

  48. Si J, Ma S, Zhang X, Gao W (2012) Adaptive rate control for high efficiency video coding. In: 2012 Visual communications and image processing. IEEE, pp 1–6

  49. Si J, Ma S, Gao W (2013a) Efficient bit allocation and ctu level rate control for high efficiency video coding. In: Picture coding symposium (PCS), 2013. IEEE, pp 89–92

  50. Si J, Ma S, Wang S, Gao W (2013b) Laplace distribution based ctu level rate control for HEVC. In: Visual communications and image processing (VCIP), 2013. IEEE, pp 1–6

  51. Sullivan G (2003)

  52. Sullivan GJ, Ohm J, Han WJ, Wiegand T (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Trans Circ Syst Vid Technol 22(12):1649–1668

    Article  Google Scholar 

  53. Sun L (2012) Rate control algorithm for high efficiency video coding (HEVC). Mphil thesis, Hong Kong University of Science and Technology

  54. Sun Y, Ahmad I (2004) A robust and adaptive rate control algorithm for object-based video coding. IEEE Trans Circ Syst Vid Technol 14 (10):1167–1182

    Article  Google Scholar 

  55. Sun L, Au OC, Dai W, Guo Y, Zou R (2012) An adaptive frame complexity based rate quantization model for intra-frame rate control of high efficiency video coding (HEVC). In: Signal & information processing association annual summit and conference (APSIPA ASC), 2012 Asia-Pacific. IEEE, pp 1–6

  56. Sze V, Budagavi M, Sullivan GJ (2014) High efficiency video coding (HEVC). In: Integrated circuit systems, algorithms and architectures, vol 39. Springer, p 40

  57. Tourapis AM, Au OC, Liou ML, Shen G, Ahmad I (2000) Optimizing the mpeg-4 encoder-advanced diamond zonal search. In: 2000 IEEE International symposium on circuits and systems (ISCAS), vol 3, pp 674–677

  58. Unterweger A, Thoma H (2007) The influence of bit rate allocation to scalability layers on video quality in H. 264 SVC. In: Picture coding symposium (PCS2007), vol 1

  59. Vetro A, Sun H, Wang Y (1999) MPEG-4 rate control for multiple video objects. IEEE Trans Circ Syst Vid Technol 9(1):186–199

    Article  Google Scholar 

  60. Wang M, Yan B (2009) Lagrangian multiplier based joint three-layer rate control for H. 264/AVC. IEEE Signal Process Lett 16(8):679–682

    Article  Google Scholar 

  61. Wang M, Ngan KN, Li H (2015) An efficient frame-content based intra frame rate control for high efficiency video coding. IEEE Signal Process Lett 22 (7):896–900. https://doi.org/10.1109/LSP.2014.2377032

    Article  Google Scholar 

  62. Wang M, Cheng B, Yuen C (2018) Joint coding-transmission optimization for a video surveillance system with multiple cameras. IEEE Trans Multimed 20(3):620–633. https://doi.org/10.1109/TMM.2017.2748459

    Article  Google Scholar 

  63. Wang S, Ma S, Wang S, Zhao D, Gao W (2013a) Rate-GOP based rate control for high efficiency video coding. IEEE J Select Topics Signal Process 7(6):1101–1111

    Article  Google Scholar 

  64. Wang S, Ma S, Zhang L, Wang S, Zhao D, Gao W (2013b) Multi layer based rate control algorithm for HEVC. In: 2013 IEEE International symposium on circuits and systems (ISCAS), IEEE, pp 41–44

  65. Wang X, Kwong S, Zhang Y (2013c) Applying game theory to rate control optimization for hierarchical B-pictures. IEEE Trans Broadcast 59 (4):591–601

    Article  Google Scholar 

  66. Wang X, Kwong S, Xu L, Zhang Y (2014) Generalized nash bargaining solution to rate control optimization for spatial scalable video coding. IEEE Trans Image Process 23(9):4010–4021

    Article  MathSciNet  MATH  Google Scholar 

  67. Wiegand T, Schwarz H, Joch A, Kossentini F, Sullivan GJ (2003) Rate-constrained coder control and comparison of video coding standards. IEEE Trans Circ Syst Vid Technol 13(7):688–703

    Article  Google Scholar 

  68. Wu W, Liu J, Feng L (2016) Novel rate control scheme for low delay video coding of HEVC. ETRI J 38(1):185–194

    Article  Google Scholar 

  69. Wu Z, Xie S, Zhang K, Wu R (2011) Rate control in video coding. In: Recent advances on video coding. InTech

  70. Xu S, Yu M, Fang S, Peng Z, Wang X, Jiang G (2015) New rate control optimization algorithm for HEVC aiming at discontinuous scene. WSEAS Trans Comput 14:598–606

    Google Scholar 

  71. Yang H, Shen L, Yang Y, Lin W (2020) A novel rate control scheme for video coding in HEVC-SCC. IEEE Trans Broadcast 66(2):333–345. https://doi.org/10.1109/TBC.2019.2954062

    Article  Google Scholar 

  72. Yang L, Zhang L, Ma S, Zhao D (2009) A ROI quality adjustable rate control scheme for low bitrate video coding. In: Picture coding symposium, 2009. PCS 2009. IEEE, pp 1–4

  73. Yang Z, Song L, Luo Z, Wang X (2014) Low delay rate control for HEVC. In: 2014 IEEE international symposium on IEEE broadband multimedia systems and broadcasting (BMSB), pp 1–5

  74. Zhao JA, Li B, Kok CW, Ahmad I (2004) Mpeg-4 video transmission over wireless networks: a link level performance study. Wirel Netw 10 (2):133–146

    Article  Google Scholar 

  75. Zheng S, Zhang X, Chen J, Kuo Y (2019) A high-efficiency compressed sensing-based terminal-to-cloud video transmission system. IEEE Trans Multimed 21(8):1905–1920. https://doi.org/10.1109/TMM.2019.2891415

    Article  Google Scholar 

  76. Zhou M, Wei X, Wang S, Kwong S, Fong CK, Wong PHW, Yuen WYF, Gao W (2019) SSIM-based global optimization for ctu-level rate control in HEVC. IEEE Trans Multimed 21(8):1921–1933. https://doi.org/10.1109/TMM.2019.2895281

    Article  Google Scholar 

  77. Zhou Y, Tian L, Ning X (2013) Intra frame constant rate control scheme for high efficiency video coding. In: 2013 international conference on IEEE computing, networking and communications (ICNC), pp 648–652

Download references

Author information

Authors and Affiliations

  1. Department of CSE, University of Texas at Arlington, Arlington, TX, USA

    Ishfaq Ahmad & Saifullah Khalid

  2. Adobe, San Jose, CA, USA

    Viswanathan Swaminathan

  3. Airforce Research Labs, Rome, NY, USA

    Alex Aved

Authors
  1. Ishfaq Ahmad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Viswanathan Swaminathan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alex Aved
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Saifullah Khalid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ishfaq Ahmad.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmad, I., Swaminathan, V., Aved, A. et al. An overview of rate control techniques in HEVC and SHVC video encoding. Multimed Tools Appl 81, 34919–34950 (2022). https://doi.org/10.1007/s11042-021-11249-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11249-5

Keywords

Search

Navigation