Skip to main content
SpringerLink
Log in

A region-based intra-frame rate control scheme by jointing inter-frame dependency and inter-frame correlation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

During I frame switching, the subjective quality between I frame and P frames usually have obvious fluctuation due to different coding methods. The periodic temporal visual fluctuation will cause video flicker. According to extensive experiments, we observe that I frame flicker possess a strong regional characteristics and different region have different degree of flicker. Based on this observation, a region-based I frame rate control scheme is proposed to suppress I frame flicker according to the different characteristics of the moving and non-moving regions. Firstly, by jointly considering the inter-frame dependency between I frame and subsequent un-encoded P frames and the inter-frame correlation between I frame and previous encoded P frame, an optimization model is proposed to achieve the optimal QPs for different regions. Secondly, a region-based inter-frame dependency model is proposed to separately describe the inter-frame dependency of different regions, which can accurately describe their description of the inter-frame dependency. The experimental results demonstrate that the proposed scheme can efficiently suppress I frame flicker and maintain the smoothness of subjective quality. Moreover, the proposed scheme can achieve a PSNR gain by 0.26 dB on average when compared with the rate control scheme adopted by the HEVC reference software HM15.0.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Becker A, Chan W, Poulouin D (2004) Flicker reduction in intra frame codecs. IEEE data compression conference (DCC), pp 251–261

  2. Bross B, Han WJ, Ohm JR, Sullivan GJ and Wiegand T (2012) High efficiency video coding (HEVC) text specification draft 8. JCT-VC document, JCTVC-J1003

  3. Changuel N, Sayadi B (2010) H.264/AVC inter-frame rate-distortion dependency analysis based on independent regime-switching AR models. IEEE international conference on acoustics speech and signal processing (ICASSP), pp 914--917

  4. Chao J, Huitl R, Steinbach E (2015) A novel rate control framework for SIFT/SURF feature preservation in H.264/AVC video compression. IEEE Trans Circuits Syst Video Technol (TCSVT 25(6):958–972

    Article  Google Scholar 

  5. Chen Z, Guillemot C (2010) Perceptually-friendly H.264/AVC video coding based on foveated just-noticeable-distortion model. IEEE Trans Circuits Syst Video Technol (TCSVT) 20(6):806–819

    Article  Google Scholar 

  6. Choi H, Nam J, Yoo J, Sim D and Bajić VI (2012) Rate control based on unified RQ model for HEVC. Joint collaborative team on video coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IECJTC1/SC29/WG118th meeting

  7. Chono K, Senda Y, Miyamoto Y (2006) Detented quantization to suppress flicker artifacts in periodically inserted Intra-coded pictures. In: IEEE international conference on image processing,H.264 video coding, (ICIP), pp 1713--1716

  8. Chun SS, Kim JR (2006) Intra prediction mode selection for flicker reduction in H.264/AVC. IEEE Trans Consum Electron (TCE) 52(4):1303–1310

    Article  Google Scholar 

  9. Chun S, Kim JR, Sull S (2006) Flicker reduction in intra coded frames of H.264/AVC. In: IEEE international conference on image processing (ICIP), pp 2025--2028

  10. Coding of audio-visual objects—Part 2: Visual (1999) in ISO/IEC 14 496–2 (MPEG-4 Visual Version 1), Apr

  11. H.264/AVC Reference Software (JM16.0) (2009) Available: http://iphome.hhi.de/suehring/tml

  12. HEVC Test Model (HM15.0) (2014) Available: http://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware

  13. Hong SH, Yoo SJ, Lee SW et al (2003) Rate control of MPEG video for consistent picture quality. IEEE Trans Broadcast (TB) 49(1):1–13

    Article  Google Scholar 

  14. Hu HM, Li B, Lin W, Li W, Sun MT (2012) Region-based rate control for H.264/AVC for Low Bit-rate applications. IEEE Trans Circuits Syst Video Technol (TCSVT) 22(11):1564–1576

    Article  Google Scholar 

  15. Jia Y, Lin W, Kassim AA (2006) Estimating just-noticeable distortion for video. IEEE Trans Circuits Syst Video Technol (TCSVT) 16(7):820–829

    Article  Google Scholar 

  16. Jing X, Chau L (2006) A novel Intra-rate estimation method for H.264 rate control. In: IEEE international symposium on circuits and systems (ISCAS), pp 5019–5022

  17. Jing X, Chau LP, Siu W (2008) Frame complexity-based rate-quantization model for H.264/AVC Intraframe rate control. IEEE Signal Process Lett 15:373–376

    Article  Google Scholar 

  18. JVT, Advanced Video Coding (AVC) (2004) ITU-T Rec.H.264 and ISO/IEC 14496–10 (MPEG-4 Part 10), 3rd ed

  19. JVT-E070 (2002) Flicking Reduction in All Intra Frame Coding, Switzerland

  20. JVT-G012, Li Z G, Pan F et al (2003) Adaptive basic unit layer rate control for JVT. Thailand

  21. Kuge T (2002) Wavelet picture coding and its several problems of the application to the interlace HDTV and the ultra-high definition images. In: IEEE international conference on image processing (ICIP), pp III-217-220

  22. Lee B, Kin M, Nguyen TQ (2014) A frame-level rate control scheme based on texture and Nontexture rate models for high efficiency video coding. IEEE Trans Circuits Syst Video Technol (TCSVT) 24(3):465–479

    Article  Google Scholar 

  23. Lee J, Shin I, Park H (2006) Adaptive intra-frame assignment and bit-rate estimation for variable GOP length in H.264. IEEE Trans Circuits Syst Video Technol (TCSVT) 16(10):1271–1279

    Article  Google Scholar 

  24. Lee YG, Song BC (2009) An intra-frame rate control algorithm for ultralow delay H.264/advanced video coding (AVC). IEEE Trans Circuits Syst Video Technol (TCSVT) 19(5):747–752

    Article  Google Scholar 

  25. Leontaris A, Tonomura Y, Nakachi T et al (2006) Rate control for flicker artifact suppression in motion JPEG2000. IEEE international conference on acoustics, speech, and signal processing (ICASSP), pp II-41-44

  26. Leontaris A, Tonomura Y, Nakachi T (2007) Flicker suppression in JPEG2000 using segmentation-based adjustment of block truncation lengths. In: IEEE international conference on acoustics, speech, and signal processing (ICASSP), pp 1117–1120

  27. Li B (2012) Rate control by R-lambda model for HEVC. In: Proceedings of the 11th meeting joint collaborative team video coding (JCT-VC) of ITU-T1/SC 29/WG 11, China

  28. Li X, Hutter A, Kaup A (2009) One-pass frame level budget allocation in video coding using inter-frame dependency, IEEE Intl. Multimedia Signal Processing (MMSP), pp: 1–6

  29. Li Y, Jia H, Zhu C, Yang M, Xie X and Gao W (2014) Window-based rate control for video quality optimization with a novel INTER-dependent rate-distortion model. In: Signal processing: image communication (SPIC)

  30. Li B, Li H, Li L, Zhang J (2014) λ domain rate control algorithm for high efficiency video coding. IEEE Trans Image Process (TIP) 23(9):3841–3854

    Article  MathSciNet  Google Scholar 

  31. Li X, Wien M, Ohm JR (2011) Rate-complexity-distortion optimization for hybrid video coding. IEEE Trans Circuits Syst Video Technol (TCSVT) 21(7):957–970

    Article  Google Scholar 

  32. Li S, Xu M, Deng X, Wang Z (2015) Weight-based R- lambda rate control for perceptual HEVC coding on conversational videos. Signal Process Image Commun 38:127–140

    Article  Google Scholar 

  33. Lin LJ (1997) Video bit-rate control with spline approximated rate-distortion characteristics. University of Southern California, California

    Google Scholar 

  34. Lin LJ, Ortega A (1998) Bit-rate control using piecewise approximated rate-distortion characteristics. IEEE Trans Circuits Syst Video Technol (TCSVT) 8:446–459

    Article  Google Scholar 

  35. Liu S, Kuo CJ (2005) Joint temporal-spatial bit allocation for video coding with dependency. IEEE Trans Circuits Syst Video Technol (TCSVT) 15:15–26

    Article  Google Scholar 

  36. Matsumura A, Naito S, Kawada R (2005) Effective rate control method for minimizing temporal fluctuations in picture quality applicable for MPEG-4 AVC/H.264 encoding. IEEE international conference on image processing (ICIP), pp 569--572

  37. Nguyen TQ, Yea S, Vetro A (2009) Adaptive fuzzy filtering for artifact reduction in compressed images and videos. IEEE Trans Image Process (TIP) 18:1166–1178

    Article  MathSciNet  Google Scholar 

  38. Pan F, Li ZG, Lim KP (2004) Adaptive intra-frame quantization for very low bit rate video coding. In: IEEE international symposium on circuits and systems (ISCAS), pp 781–784

  39. Pang C, Au OC, Dai J (2011) Frame-level dependent bit allocation via geometric programming. In: IEEE international symposium on circuits and systems (ISCAS), pp 2149--2152

  40. Pang C, Au OC, Zou F (2011) An analytical framework for frame-level dependent bit allocation in hybrid video coding, IEEE Intl. Workshop on Multimedia Signal Processing (MMSP), pp: 1–4

  41. Seo CW, Moon JH, Han JK (2013) Rate control for consistent objective quality in high efficiency video coding. IEEE Trans Image Process (TIP) 22(6):2442–2454

    Article  MathSciNet  Google Scholar 

  42. Sullivan GJ, Ohm J, Han WJ, Wigand T (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Trans Circuits Syst Video Technol (TCSVT) 22(12):1649–1668

    Article  Google Scholar 

  43. Tang CW (2007) Spatiotemporal visual considerations for video coding. IEEE Trans Multimedia (TMM) 9(2):231–238

    Article  Google Scholar 

  44. Tang CW, Chen CH, Yu YH et al (2006) Visual sensitivity guided bit allocation for video coding. IEEE Trans Multimedia (TMM) 8(1):11–18

    Article  Google Scholar 

  45. Tian L, Sun Y, Ahmad I (2009) Effective intra-only rate control for H.264/AVC. In: IEEE international conference on image processing (ICIP), pp 3445–3448

  46. Tsai WJ, Chou TL (2010) Scene change aware intra-frame rate control for H.264/AVC. IEEE Trans Circuits Syst Video Technol (TCSVT) 20(12):1882–1886

    Article  Google Scholar 

  47. Wang S, Ma S, Wang S, Zhao D, Gao W (2013) Rate-GOP based rate control for high efficiency video coding. IEEE J Sel Top Sign Proces 7(6):1101–1111

    Article  Google Scholar 

  48. Wang M, Ngan KN, Li H (2015) An efficient frame-content based intra frame rate control for high efficiency video coding. IEEE Signal Process Lett 22(7):896–900

    Article  Google Scholar 

  49. Wang Y, Sun J, Ma S et al (2009) Theoretic analysis of Inter frame dependency in video coding. In: 9th Pacific Rim conference on multimedia (PCM) [C], pp 935–939

  50. Wang P, Zhang Y, Hu HM (2013) Region classification-based rate control for flicker suppression of I frames in HEVC. In: IEEE international conference on image processing (ICIP), pp 1986–1990

  51. Wei Z, Ngan KN (2009) Spatio-temporal just noticeable distortion profile for grey scale image/video in DCT domain. IEEE Trans Circuits Syst Video Technol (TCSVT) 19(3):337–346

    Article  Google Scholar 

  52. Wiegand T, Sullivan GJ, Bjøntegaard G, Luthra A (2003) Overview of the H.264/AVC video coding standard. IEEE Trans Circuits Syst Video Technol (TCSVT) 13(7):560–576

    Article  Google Scholar 

  53. Yan Y, Sun K (2012) Joint complexity estimation of I-frame and P frame for H.264/AVC rate control. IEEE Trans Circuits Syst Video Technol (TCSVT) 22(5):790–798

    Article  Google Scholar 

  54. Yan B, Wang M (2009) Adaptive distortion-based intra-rate estimation for H.264/AVC rate control. IEEE Signal Process Lett (SPL) 16(3):145–148

    Article  Google Scholar 

  55. Yang H, Boyce JM (2008) Effective flicker removal from periodic Intra frames and accurate flicker measurement. In: IEEE international conference on image processing (ICIP), pp 2868–2871

  56. Yang JX, Wu HR (2010) Robust filtering technique for reduction of temporal fluctuation. IEEE Trans Circuits Syst Video Technol (TCSVT) 20(3):458–462

    Article  Google Scholar 

  57. Zhou M, Hu HM and Zhang Y (2014) Region-based intra-frame rate control Scheme for high efficiency video coding. Asian-Pacific signal and information processing association annual summit and conference (APSIPA ASC)

  58. Zhou Y, Sun Y, Feng Z et al (2009) New rate-distortion modeling and efficient rate control for H.264/AVC video coding. Signal Process Image Commun (SPIC) 24:345–356

    Article  Google Scholar 

Download references

Acknowledgments

This work was partially supported by the National Natural Science Foundation of China (No. 61370121), and the National Hi-Tech Research and Development Program (863 Program) of China (No.2014AA015102).

Author information

Authors and Affiliations

  1. Beijing Key Laboratory of Digital Media, School of Computer Science and Engineering, Beihang University, Beijing, 100191, China

    Hai-Miao Hu, Mingliang Zhou & Naiyu Yin

  2. State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, 100191, China

    Hai-Miao Hu

  3. Beijing Institute of Graphics, Beijing, 100029, China

    Yang Liu

Authors
  1. Hai-Miao Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingliang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Naiyu Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hai-Miao Hu.

Appendix: derivation of P frame error

Appendix: derivation of P frame error

Suppose that the original image of I frame is

f I (x, y), and the corresponding reconstructed image is

f ″ I (x, y). The original image of the first P frame after I frame is f P (x, y), and its predicted image and reconstructed image are f P ′(x, y) and f ″ P (x, y), respectively. Then the frame error between I frame and P frame (the error between the reconstructed frame and the original frame) can be defined as:

$$ \begin{array}{l}{E}_I={f}_I\left(x,y\right)-{f}_I^{{\prime\prime}}\left(x,y\right)\hfill \\ {}{E}_P={f}_P\left(x,y\right)-{f}_P^{{\prime\prime}}\left(x,y\right)\hfill \end{array} $$
(A-1)

where, E I is the frame error of I frame, and E P is the frame error of P frame. Assume that e P (x, y) is the residual between the predicted image and the original image, i.e., e P (x, y) = f P (x, y) − f P ′(x, y), and the quantized value is e ″ P (x, y) = Quant(e P (x, y)). The reconstructed frame of P frame can be represented as:

$$ {f}_P^{{\prime\prime}}\left(x,y\right)={f}_P^{\prime}\left(x,y\right)+{e}_P^{{\prime\prime}}\left(x,y\right) $$
(A-2)

where, the predicted image of P frame can be further represented as:

$$ {f}_P^{\prime}\left(x,y\right)={f}_I^{{\prime\prime}}\left(x+i,y+j\right) $$
(A-3)

where, (i, j) is the motion vector. Then after incorporating (A-2) and (A-3) into (A-1), the frame error of P frame can be rewritten as:

$$ \begin{array}{l}{E}_P={f}_P\left(x,y\right)-\left({f}_P^{\prime}\left(x,y\right)+{e}_P^{{\prime\prime}}\left(x,y\right)\right)\hfill \\ {}{E}_P={f}_P\left(x,y\right)-{f}_I\left(x,y\right)+\left({f}_I\left(x,y\right)-{f}_P^{\prime}\left(x,y\right)\right)-{e}_P^{{\prime\prime}}\left(x,y\right)\hfill \\ {}{E}_P={f}_P\left(x,y\right)-{f}_I\left(x,y\right)+\left({f}_I\left(x,y\right)-{f}_I^{{\prime\prime}}\left(x+i,y+j\right)\right)-{e}_P^{{\prime\prime}}\left(x,y\right)\hfill \end{array} $$
(A-4)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, HM., Zhou, M., Liu, Y. et al. A region-based intra-frame rate control scheme by jointing inter-frame dependency and inter-frame correlation. Multimed Tools Appl 76, 12917–12940 (2017). https://doi.org/10.1007/s11042-016-3666-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3666-x

Keywords

Search

Navigation