Abstract
The introduction of the new coding tools in HEVC has brought significant bitrate savings compared to the previous standard, H.264/AVC. In both standards, the mean square error (MSE) is used for measuring distortion in the rate distortion optimization process of the coding unit structure and mode selection. However, MSE is not a good measure to use for measuring visual quality as it poorly correlates with human perception. Integration of a video quality metric based on the characteristics of the Human Visual System (HVS) inside the rate distortion optimization procedure is expected to improve the compression efficiency of video coding. In this paper, the PSNR-HVS measure is used in the rate distortion optimization process for the coding unit structure and mode selection. In the first step, we find the scaling factor for the Lagrangian multiplier based on the proposed perceptual approach. In the second step, we find optimal Lagrangian multiplier depending on the quantization parameter. The compression efficiency of the proposed approach is compared to that of HEVC. Simulations prove that the proposed approach yields higher compression efficiency.
Similar content being viewed by others
References
Ahn YJ, Sim D (2019) Fast mode decision and early termination based on perceptual visual quality for HEVC encoders. J Real-Time Image Proc 16(6):1927–1942
Aswathappa BHK, Rao KR (2010) Rate-distortion optimization using structural information in H.264 strictly intra-frame encoder, in System Theory (SSST), 2010 42nd Southeastern Symposium on, pp. 367–370
Bjontegaard G (2001) Calculation of average PSNR difference between RD curves, in Proc. 13th Meeting ITU-T Q.6/SG16 VCEG , Austin, TX
Bossen F (2012) HM 9 common test conditions and software reference configurations, Joint Collaborative Team on Video Coding (JCT-VC), Document JCTVC-J1100, Stockholm
Chen Z, Guillemot C (2010) Perceptually-friendly H.264/AVC video coding based on Foveated just-noticeable-distortion model. Circuits and Systems for Video Technology, IEEE Transactions on 20:806–819
Chen J, Zheng J, He Y (2007) Macroblock-level adaptive frequency weighting for perceptual video coding. Consumer Electronics, IEEE Transactions on 53:775–781
Chikkerur S, Sundaram V, Reisslein M, Karam LJ (2011) Objective video quality assessment methods: a classification, review, and performance comparison. Broadcasting, IEEE Transactions on 57:165–182
Daugman JG (1984) Spatial visual channels in the Fourier plane. Vis Res 24:891–910
Van den Branden Lambrecht, Christian J and Verscheure O (1996) Perceptual quality measure using a spatiotemporal model of the human visual system, in Electronic Imaging: Science & Technology, pp. 450–461
DeValois RL, DeValois KK (1988) Spatial Vision. Oxford University Press
Egiazarian K, Astola J, Ponomarenko N, Lukin V, Battisti F, Carli M (2006) Two new full-reference quality metrics based on HVS, Proc. of the Second International Workshop on Video Processing and Quality Metrics, Scottsdale, USA, 4 p
Everett H III (1963) Generalized Lagrange multiplier method for solving problems of optimum allocation of resources. Oper Res 11:399–417
Fu D, Wang Y, Fan B, Ding N (2019) HEVC/H. 265 intra coding based on the human visual system. IEEE Access 7:186587–186599
Hamza R, Hassan A, Huang T,, Ke L, Yan H (2019) An efficient cryptosystem for video surveillance in the internet of things, Environment. Complexity
Harshalatha Y, Biswas PK (2018) SSIM-based joint-bit allocation for 3D video coding. Multimedia Tools and Applications 77(15):19051–19069
Huynh-Thu Q, Ghanbari M (2008) Scope of validity of PSNR in image/video quality assessment. Electron Lett 44:800–801
Huynh-Thu Q, Ghanbari M (2012) The accuracy of PSNR in predicting video quality for different video scenes and frame rates. Telecommun Syst 49:35–48
Lee C, Kwon O (2003) Objective measurements of video quality using the wavelet transform. Opt Eng 42:265–272
Liu Z, Lin TL, Chou CC (2017) HEVC coding-unit decision algorithm using tree-block classification and statistical data analysis. Multimedia Tools and Applications 76(6):9051–9072
Lukas F, Budrikis ZL (1982) Picture quality prediction based on a visual model. Communications, IEEE Transactions on 30:1679–1692
Mai Z, Yang C, Po L, Xie S (2005) A new rate-distortion optimization using structural information in H.264 I-frame encoder, in Proc. ACIVS pp. 435–441
Mai Z, Yang C, Xie S (2005) Improved best prediction mode(s) selection methods based on structural similarity in H.264 I-frame encoder, in Proc. IEEE Int. Conf. Sys. Man Cybern., pp. 2673–2678 Vol. 3.
Z. Mai, C. Yang, K. Kuang and L. Po, "A novel motion estimation method based on structural similarity for H.264 inter prediction," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, vol. 2, Feb. 2006, pp. 913–916.
McCann K, Bross B, Han W, Kim I, Sugimoto K, Sullivan G (2013) High Efficiency Video Coding (HEVC) Test Model 13 (HM 13) Encoder Description, joint collaborative team on video coding (JCT-VC), document JCTVC-O1002, Geneva
Ou T, Huang Y, Chen HH (2011) SSIM-based perceptual rate control for video coding. Circuits and Systems for Video Technology, IEEE Transactions on 21:682–691
Recommendation I (2002) "500–11,“Methodology for the Subjective Assessment of the Quality of Television Pictures,” Recommendation ITU-R BT. 500–11," ITU TelecomStandardization Sector of ITU
Rehman A, Wang Z (2012) SSIM-inspired perceptual video coding for HEVC, in Multimedia and Expo (ICME), 2012 IEEE International Conference on, pp. 497–502
Ruiz D, Fernández-Escribano G, Adzic V, Kalva H, Martínez JL, Cuenca P (2017) Fast CU partitioning algorithm for HEVC intra coding using data mining. Multimedia tools and applications 76(1):861–894
Seshadrinathan K, Bovik AC (2010) Motion tuned spatio-temporal quality assessment of natural videos. Image Processing, IEEE Transactions on 19:335–350
Sullivan GJ, Wiegand T (1998) Rate-distortion optimization for video compression. Signal Processing Magazine, IEEE 15:74–90
Sullivan GJ, Ohm J, Han W-J, Wiegand T (2012) Overview of the high efficiency video coding (HEVC) standard. Circuits and Systems for Video Technology, IEEE Transactions on 22:1649–1668
Sun X, Ma H, Zuo W, Liu M (2019) Perceptual-based HEVC intra coding optimization using deep convolution networks. IEEE Access 7:56308–56316
Sze V, Budagavi M, Sullivan G (2014) High efficiency video coding (HEVC), Springer
Valizadeh S, Nasiopoulos P, Ward R (2015) “Perceptually-friendly rate distortion optimization in high efficiency video coding," European Signal Processing Conference (EUSIPCO-2015)
Valizadeh S, Nasiopoulos P, Ward R (2016) Optimal Lagrange Multiplier in Perceptually-Friendly High Efficiency Video Coding for Mobile Applications, the International Conference on Computing, Networking and Communications (ICNC16), CNC Workshop, Hawaii
Wallace G (1991) The JPEG still picture compression standard, Comm. of the ACM, vol. 34, No.4
Wang S, Rehman A, Wang Z, Ma S, Gao W (2012) SSIM-motivated rate-distortion optimization for video coding. Circuits and Systems for Video Technology, IEEE Transactions on 22:516–529
Wang S, Rehman A, Wang Z, Ma S, Gao W (2013) Perceptual video coding based on SSIM-inspired divisive normalization. Image Processing, IEEE Transactions on 22:1418–1429
Wang G, Zhang Y, Li B, Fan R, Zhou M (2018) A fast and HEVC-compatible perceptual video coding scheme using a transform-domain Multi-Channel JND model. Multimedia Tools and Applications 77(10):12777–12803
Watson AB, Hu J, McGowan JF (2001) Digital video quality metric based on human vision. Journal of Electronic Imaging 10:20–29
Xiao F (2000) DCT-based video quality evaluation, Final Project for EE392J, vol. 769
Yang C, Wang H, Po L (2007) Improved inter prediction based on structural similarity in H.264, in Signal Processing and Communications. ICSPC 2007. IEEE International Conference on, 2007, pp. 340–343.
Yang C, Leung R, Po L, Mai Z (2009) An SSIM-optimal H.264/AVC inter frame encoder, in Intelligent Computing and Intelligent Systems, ICIS 2009. IEEE International Conference on, 2009, pp. 291–295
Yeo C, Li Tan H, Tan YH (2013) On rate distortion optimization using SSIM. Circuits and Systems for Video Technology, IEEE Transactions on 23:1170–1181
Zhao T, Zeng K, Rehman A, Wang Z (2013) On the use of SSIM in HEVC, in Signals, Systems and Computers, 2013 Asilomar Conference on, pp. 1107–1111
Acknowledgements
This work was supported by the NPRP grant # NPRP 4-463-2-172 from the Qatar National Research Fund (a member of the Qatar Foundation). The statements made herein are solely the responsibility of the authors.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Valizadeh, S., Nasiopoulos, P. & Ward, R. Improving compression efficiency of HEVC using perceptual coding. Multimed Tools Appl 80, 10235–10254 (2021). https://doi.org/10.1007/s11042-020-09442-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09442-z