Skip to main content
SpringerLink
Log in

The accuracy of PSNR in predicting video quality for different video scenes and frame rates

  • Published:
Telecommunication Systems Aims and scope Submit manuscript

Abstract

Peak Signal-to-Noise Ratio (PSNR) is widely used as a video quality metric or performance indicator. Some studies have indicated that it correlates poorly with subjective quality, whilst others have used it on the basis that it provides a good correlation with subjective data. Existing literature seems to provide conflicting evidence of the accuracy of PSNR as a video quality metric. Based on experimental results, we explain a scenario where PSNR provides a reliable indication of the variation of subjective video quality and scenarios where PSNR is not a reliable video quality metric. We show that PSNR follows a monotonic relationship with subjective quality in the case of full frame rate encoding when the video content and codec are fixed. We provide evidence that PSNR becomes an unreliable and inaccurate quality metric when several videos with different content are jointly assessed. Furthermore, PSNR is inaccurate in measuring video quality of a video content encoded at different frame rates because it is not capable of assessing the perceptual trade-off between the spatial and temporal qualities. Finally, where PSNR is not a reliable video quality metric across different video contents and frame rates, we show that a perceptual video model recently approved by the International Telecommunication Union (ITU) provides quality predictions highly correlating with subjective scores even if different video scenes coded at different frame rates are considered in the test set.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Girod, B. (1993). What’s wrong with mean-squared error? In A. B. Watson (Ed.), Digital images and human vision (pp. 207–220). Cambridge: MIT Press.

    Google Scholar 

  2. Teo, P., & Heeger, D. (1994). Perceptual image distortion. In Proceedings of SPIE human vision, visual processing and digital display V (Vol. 2179, pp. 127–141), San Jose, February 1994.

  3. Wang, Z., & Bovik, A. C. (2002). A universal image quality index. IEEE Signal Processing Letters, 9(3), 81–84.

    Article  Google Scholar 

  4. Wang, Z., & Bovik, A. (2002). Why is image quality assessment so difficult? In Proceedings of IEEE international conference on acoustics, speech, and signal processing (ICASSP) (Vol. 4, pp. 3313–3316), Orlando, May 2002.

  5. Nemethova, O., Ries, M., Siffel, E., & Rupp, M. (2002). Quality assessment for H.264 coded low-rate and low-resolution video sequences. In Third IASTED international conference on communications, Internet, and information technology (CIIT 2004) (pp. 136–140), St. Thomas, US Virgin Islands, 22–24 January 2004.

  6. Richardson, I. (2004). H.264 and MPEG-4 video compression: video coding for next-generation multimedia. New York: Wiley.

    Google Scholar 

  7. Wolf, S., Pinson, M., Webster, A., Cermak, G., & Tweedy, E. P. (1996). Objective and subjective measures of MPEG video quality. ITS/NTIA, Tech. Rep. ANSI T1A1.5/96-121, October 1996.

  8. Masry, M., & Hemami, S. (2001). An analysis of subjective quality in low bit rate video. In Proceedings of IEEE international conference on image processing (Vol. 1, pp. 465–468), October 2001.

  9. Oelbaum, T., Diepold, K., & Zia, W. (2007). A generic method to increase the prediction accuracy of visual quality metrics. In Proceedings of picture coding symposium, Lisbon, November 2007.

  10. Video Quality Experts Group (2008). Draft final report from the Video Quality Experts Group on the validation of objective models of multimedia quality assessment Phase I, May 2008, available at http://www.vqeg.org.

  11. Reed, E., & Dufaux, F. (2001). Constrained bit-rate control for very low bit-rate streaming-video applications. IEEE Transactions on Circuits and Systems for Video Technology, 11(7), 882–889.

    Article  Google Scholar 

  12. Song, H., & Kuo, C. J. (2001). Rate control for low-bit-rate video via variable-encoding frame rates. IEEE Transactions on Circuits and Systems for Video Technology, 11(4), 512–521.

    Article  Google Scholar 

  13. Feghali, R., Speranza, F., Wang, D., & Vincent, A. (2007). Video quality metric for bit rate control via joint adjustment of quantization and frame rate. IEEE Transactions on Broadcasting, 53(1), 441–446.

    Article  Google Scholar 

  14. International Telecommunication Union. (2004). Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference. ITU-T Rec.J.144, March 2004.

  15. International Telecommunication Union. (2004). Objective perceptual video quality measurement techniques for standard definition digital broadcast television in the presence of a full reference. ITU-R Rec. BT.1683, June 2004.

  16. Video Quality Experts Group. Final report from the Video Quality Experts Group on FRTV Phase 2: Test validation of objective models of video quality assessment, August 2003. Available at http://www.vqeg.org.

  17. Wiegand, T., Schwarz, H., Joch, A., Kossentini, F., & Sullivan, G. (2003). Rate-constrained coder control and comparison of video coding standards. IEEE Transactions on Circuits and Systems for Video Technology, 13(7), 688–703.

    Article  Google Scholar 

  18. Ichigaya, A., Kurozumi, M., Hara, N., Nishida, Y., & Nakasu, E. (2006). A method of estimating coding PSNR using quantized DCT coefficients. IEEE Transactions on Circuits and Systems for Video Technology, 16(2), 251–259.

    Article  Google Scholar 

  19. Eden, A. (2007). No-reference estimation of the coding PSNR for H.264-coded sequences. IEEE Transactions on Consumer Electronics, 53(2), 667–674.

    Article  Google Scholar 

  20. Huynh-Thu, Q., & Ghanbari, M. (2008). Scope of validity of PSNR in image/video quality assessment. IET Electronic Letters, 44(13), 800–801.

    Article  Google Scholar 

  21. International Telecommunication Union. (2008). Subjective video quality assessment methods for multimedia applications. ITU-T Rec. P.910, April 2008.

  22. International Telecommunication Union. (2008). Objective perceptual multimedia video quality measurement in the presence of a full reference. ITU-T Rec. J.247, August 2008.

  23. Yadavalli, G., Masry, M., & Hemami, S. (2003). Frame rate preferences in low bit rate video. In Proceedings of IEEE international conference on image processing (Vol. 1, pp. 441–444), September 2003.

Download references

Author information

Authors and Affiliations

  1. Psytechnics Ltd, 23 Museum Street, IP1 1HN, Ipswich, Suffolk, UK

    Quan Huynh-Thu

  2. University of Essex, Wivenhoe Park, CO4 3SQ, Colchester, Essex, UK

    Mohammed Ghanbari

Authors
  1. Quan Huynh-Thu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammed Ghanbari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Quan Huynh-Thu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Huynh-Thu, Q., Ghanbari, M. The accuracy of PSNR in predicting video quality for different video scenes and frame rates. Telecommun Syst 49, 35–48 (2012). https://doi.org/10.1007/s11235-010-9351-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11235-010-9351-x

Keywords

Search

Navigation