Skip to main content
SpringerLink
Log in

Video Rate Controller Development Through Neuro-Fuzzy Quantization Parameter Modifiers

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

In this paper, a video Rate Controller (RC) is proposed for the Versatile Video Coding (H.266/VVC) standard. This RC modifies the Quantization Parameter (QP) to keep the buffer state stable and produce the bandwidth-compliant rate simultaneously. It conforms with the specifications of the Variable Bit-Rate (VBR) applications. Intra-frames have high bit-production, which leads to a sudden increase in the rate. Hence, the Group-of-Pictures (GOPs) are categorized into intra-included and intra-excluded. For each type, a distinct neuro-fuzzy QP modifier is designed. The parameters of the neuro-fuzzy QP modifiers are achievable through the training process. However, two data sets are required to train each of the modifiers. These data sets are constructed by the data acquired from the experiments and taking advantage of the q-learning algorithm. Experiments show that the proposed RC achieves the target rate and maintains the buffer state stable. The rate-distortion analysis reveals that the proposed RC has a 2.72% rate-saving capability compared with the one implemented in the reference software of the H.266/VVC. In the case of 4 K sequences, the proposed method saves 5.15% of the rate compared to the rate control algorithm implemented in the reference software.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5

Similar content being viewed by others

Availability of Data and Material

Not Applicable.

Code Availability

Not Applicable.

References

  1. Hamidouche, W., Biatek, T., Abdoli, M., Francois, E., Pescador, F., Radosavljevic, M., Menard, D., Raulet, M. (2022). Versatile video coding standard: A review from coding tools to consumers deployment. IEEE Consumer Electronics Magazine.

  2. Ahmad, I., Swaminathan, V., Aved, A., Khalid, S. (2021). An overview of rate control techniques in HEVC and SHVC video encoding. Multimedia Tools and Applications, 1–32.

  3. Hou, Y., Wang, P., Xiang, W., Gao, Z., & Hou, C. (2015). A novel rate control algorithm for video coding based on fuzzy-PID controller. Signal, Image and Video Processing, 9(4), 875–884.

    Article  Google Scholar 

  4. Fani, D., & Rezaei, M. (2016). A GOP-level fuzzy rate control algorithm for high-delay applications of HEVC. Signal, Image and Video Processing, 10(7), 1183–1191.

    Article  Google Scholar 

  5. Fani, D., & Rezaei, M. (2017). Novel PID-Fuzzy Video Rate Controller for High-Delay Applications of the HEVC Standard. IEEE Transactions on Circuits and Systems for Video Technology, 28(6), 1379–1389.

    Article  Google Scholar 

  6. Shojaei, M., Rezaei, M. (2020). FJND-based fuzzy rate control of scalable video for streaming applications. Multimedia Tools and Applications, 1–21.

  7. Nakhaei, A., Rezaei, M. (2020). Scene-level two-pass video rate controller for H. 265/HEVC standard. Multimedia Tools and Applications, 1–16.

  8. Ross, T. J. (2005). Fuzzy logic with engineering applications. John Wiley & Sons.

  9. Jang, J.-S. (1993). ANFIS: Adaptive-network-based fuzzy inference system. IEEE transactions on systems, man, and cybernetics, 23(3), 665–685.

    Article  Google Scholar 

  10. Wien, M. (2015). High efficiency video coding. Coding Tools and Specification, 133–160.

  11. Busoniu, L., Babuska, R., De Schutter, B., Ernst, D. (2010) Reinforcement learning and dynamic programming using function approximators. 39. CRC press.

  12. Li, B., Li, H., Li, L., & Zhang, J. (2014). λ-domain rate control algorithm for High Efficiency Video Coding. IEEE transactions on Image Processing, 23(9), 3841–3854.

    Article  MathSciNet  MATH  Google Scholar 

  13. Li, Q., Nie, J., Yu, D. (2021). Space-domain-based CTU layer rate control for HEVC. Signal, Image and Video Processing, 1–8.

  14. Zhao, Z., Xiong, S., Sun, W., He, X., Zhang, F. (2020). An improved R-λ rate control model based on joint spatial-temporal domain information and HVS characteristics. Multimedia Tools and Applications, 1–22.

  15. Lin, H., Li, X., Gao, M., Deng, K., & Xu, Y. (2022). Perceptual importance analysis-based rate control method for HEVC. Multimedia Tools and Applications, 81(9), 12495–12518.

    Article  Google Scholar 

  16. Li, Y., & Mou, X. (2021). Joint Optimization for SSIM-Based CTU-Level Bit Allocation and Rate Distortion Optimization. IEEE Transactions on Broadcasting, 67(2), 500–511.

    Article  Google Scholar 

  17. Yuan, H., Wang, Q., Liu, Q., Huo, J., Li, P. (2021). Hybrid Distortion-Based Rate-Distortion Optimization and Rate Control for H. 265/HEVC. IEEE Transactions on Consumer Electronics, 67(2), 97–106.

  18. Lim, W., Sim, D. (2020). A perceptual rate control algorithm based on luminance adaptation for HEVC encoders. Signal, Image and Video Processing, 1–9.

  19. Xiang, G., Zhang, X., Huang, X., Yang, F., Zhu, C., Jia, H., & Xie, X. (2021). Perceptual Quality Consistency Oriented CTU Level Rate Control for HEVC Intra Coding. IEEE Transactions on Broadcasting, 68(1), 69–82.

    Article  Google Scholar 

  20. Marzuki, I., Lee, J., Wiratama, W., & Sim, D. (2021). Deep convolutional feature-driven rate control for the HEVC encoders. IEEE Access, 9, 162018–162034.

    Article  Google Scholar 

  21. Wang, T., Li, F., & Cosman, P. C. (2022). Learning-based rate control for video-based point cloud compression. IEEE Transactions on Image Processing, 31, 2175–2189.

    Article  Google Scholar 

  22. Singhadia, A., Pati, P.S., Singhal, C., Chakrabarti, I. (2022). Efficient HEVC Encoding to Meet Bitrate and PSNR Requirements Using Parametric Modeling. Circuits, Systems, and Signal Processing, 1–33.

  23. Esmaeeli, H., & Rezaei, M. (2022). A content-based intra rate-distortion model for HEVC-SCC. Multimedia Tools and Applications, 81(12), 16515–16536.

    Article  Google Scholar 

  24. Esmaeeli, H., & Rezaei, M. (2022). Proportional–integral–derivative-based intra rate controller for low-delay applications of high efficiency video coding screen content coding. Journal of Electronic Imaging, 31(2), 023029.

    Article  Google Scholar 

  25. Zhou, M., Wei, X., Kwong, S., Jia, W., & Fang, B. (2020). Rate control method based on deep reinforcement learning for dynamic video sequences in HEVC. IEEE Transactions on Multimedia, 23, 1106–1121.

    Article  Google Scholar 

  26. Sun, Y., Childers, R., Hilton, J. (2021). New bufferless rate control for high efficiency video coding. Multimedia Tools and Applications, 1–16.

  27. Raufmehr, F., Salehi, M. R., & Abiri, E. (2021). A frame-level MLP-based bit-rate controller for real-time video transmission using VVC standard. Journal of Real-Time Image Processing, 18(3), 751–763.

    Article  Google Scholar 

  28. Raufmehr, F., Salehi, M. R., & Abiri, E. (2021). A neural network-based video bit-rate control algorithm for variable bit-rate applications of versatile video coding standard. Signal Processing: Image Communication, 96, 116317.

    Google Scholar 

  29. Raufmehr, F., Salehi, M.R., Abiri, E. (2022). Overcoming the practical restrictions in H. 266/VVC-based video communication systems by a PI bit rate controller. Multimedia Systems, 1–17.

  30. Yiming Li, Z. C. (2018). Rate Control for VVC JVET-K0390(ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11).

  31. Hyun, M. H., Lee, B., & Kim, M. (2020). A Frame-Level Constant Bit-Rate Control Using Recursive Bayesian Estimation for Versatile Video Coding. IEEE Access, 8, 227255–227269.

    Article  Google Scholar 

  32. Liu, F., & Chen, Z. (2021). Multi-Objective Optimization of Quality in VVC Rate Control for Low-Delay Video Coding. IEEE Transactions on Image Processing, 30, 4706–4718.

    Article  MathSciNet  Google Scholar 

  33. Chen, Y., Kwong, S., Zhou, M., Wang, S., Zhu, G., Wang, Y. (2020). Intra frame rate control for versatile video coding with quadratic rate-distortion modelling. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4422–4426. IEEE.

  34. Ren, G., et al. (2021). Reinforcement Learning based ROI Bit Allocation for Gaming Video Coding in VVC. 2021 International Conference on Visual Communications and Image Processing (VCIP), 1–5.

  35. Ren, G., Liu, Z., Chen, Z., Liu, S. (2021). Reinforcement Learning based ROI Bit Allocation for Gaming Video Coding in VVC. In: 2021 International Conference on Visual Communications and Image Processing (VCIP), pp. 1–5. IEEE.

  36. Mao, Y., Wang, M., Wang, S., Kwong, S. (2021). High Efficiency Rate Control for Versatile Video Coding Based on Composite Cauchy Distribution. IEEE Transactions on Circuits and Systems for Video Technology.

  37. Helmrich, C.R., Zupancic, I., Brandenburg, J., George, V., Wieckowski, A., Bross, B. (2021). Visually Optimized Two-Pass Rate Control for Video Coding Using the Low-Complexity XPSNR Model. In: 2021 International Conference on Visual Communications and Image Processing (VCIP), pp. 1–5. IEEE.

  38. Bossen, F. (2020). VTM common test conditions and software reference configurations for SDR video. Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29 JVET-T2010.

  39. The Ultra Video Group (UVG) test sequences. Retrieved from https://ultravideo.fi/#testsequences

  40. Akramullah, S. (2014). Digital video concepts, methods, and metrics: quality, compression, performance, and power trade-off analysis: Apress.

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Electronics Engineering, Shiraz University of Technology, Shiraz, Iran

    Farhad Raufmehr & Mohammad Reza Salehi

Authors
  1. Farhad Raufmehr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Reza Salehi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Farhad Raufmehr.

Ethics declarations

Conflicts of Interest/Competing Interests

Not Applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Raufmehr, F., Salehi, M.R. Video Rate Controller Development Through Neuro-Fuzzy Quantization Parameter Modifiers. J Sign Process Syst 95, 751–764 (2023). https://doi.org/10.1007/s11265-023-01877-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-023-01877-5

Keywords

Search

Navigation