Skip to main content
Springer Nature Link
Log in

Digital Video Quality Assessment Algorithms

  • Chapter
  • First Online:
Handbook of Multimedia for Digital Entertainment and Arts

Abstract

In this chapter we first describe some HVS-based approaches which try to model the visual processing stream described above, since these approaches were originally used to predict visual quality. We then describe recently proposed structural and information-theoretic approaches and feature-based approaches which are commonly used. Further, we describe recent motion-modeling based approaches, and detail performance evaluation and validation techniques for VQA algorithms. Finally, we touch upon some possible future directions for research on VQA and conclude the chapter.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The receptive field of a neuron is its response to visual stimuli, which may depend on spatial frequency, movement, disparity or other properties. As used here, the receptive field response may be viewed as synonymous with the signal processing term impulse response.

References

  1. Z. Wang and A. C. Bovik, Modern Image Quality Assessment. New York: Morgan and Claypool Publishing Co., 2006.

    Google Scholar 

  2. A. K. Moorthy and A. C. Bovik, “Perceptually Significant Spatial Pooling techniques for Image quality assessment ,” in SPIE Conference on Human Vision and Electronic Imaging, Jan. 2009.

    Google Scholar 

  3. “Methodology for the subjective assessment of the quality of television pictures,” ITU-R Recommendation BT.500–11.

    Google Scholar 

  4. B. Hiremath, Q. Li and Z. Wang “Quality-aware video,”IEEE International Conference on Image Processing, San Antonio, TX, Sept. 16–19, 2007.

    Google Scholar 

  5. H. R. Sheikh, A. C. Bovik, and L. Cormack, “No-reference quality assessment using natural scene statistics: JPEG2000,”Image Processing, IEEE Transactions on, vol. 14, no. 11, pp. 1918–1927, 2005.

    Article  Google Scholar 

  6. C. M. Liu, J. Y. Lin, K. G. Wu and C. N. Wang, “Objective image quality measure for block-based DCT coding,” IEEE Trans. Consum. Electron., vol. 43, pp. 511–516, 1997.

    Article  Google Scholar 

  7. Z. Wang, A. C. Bovik, and B. L. Evans, “Blind measurement of blocking artifacts in images,” in IEEE Intl. Conf. Image Proc, 2000.

    Google Scholar 

  8. X. Li, “Blind image quality assessment”, IEEE International Conference on Image Processing, New York, 2002.

    Google Scholar 

  9. Patrick Le Callet, Christian Viard-Gaudin, Stéphane Péchard and Emilie Caillault, “No reference and reduced reference video quality metrics for end to end QoS monitoring”, Special Issue on multimedia Qos evaluation and management technologies, E89, (2), Pages: 289–296, February 2006.

    Google Scholar 

  10. W. S. Geisler and M. S. Banks, “Visual performance,” in Handbook of Optics, M. Bass, Ed. McGraw-Hill, 1995.

    Google Scholar 

  11. B. A. Wandell, Foundations of Vision. Sunderland, MA: Sinauer Associates Inc., 1995.

    Google Scholar 

  12. N. C. Rust, V Mante, E. P. Simoncelli, and J. A. Movshon, “How MT cells analyze the motion of visual patterns ”, Nature Neuroscience, vol.9(11), pp. 1421–1431, Nov 2006.

    Article  Google Scholar 

  13. Z. Wang, G. Wu, H. R. Sheikh, E. P. Simoncelli, E.-H. Yang and A. C. Bovik, ”Quality -aware images” IEEE Transactions on Image Processing, vol. 15, no. 6, pp. 1680–1689, June 2006.

    Article  Google Scholar 

  14. R. T. Born and D. C. Bradley, “Structure and function of visual area MT,” Annual Rev Neuroscience, vol. 28, pp. 157–189, 2005.

    Article  Google Scholar 

  15. M. A. Smith, N. J. Majaj, and J. A. Movshon, “Dynamics of motion signaling by neurons in macaque area MT,” Nature Neuroscience, vol. 8, no. 2, pp. 220–228, Feb. 2005.

    Article  Google Scholar 

  16. S. Daly, “The visible differences predictor: an algorithm for the assessment of image fidelity,” in Digital Images and Human Vision (A. B. Watson, ed.), pp. 179–206, Cambridge, MA: The MIT Press, 1993.

    Google Scholar 

  17. J. Lubin, “The use of psychophysical data and models in the analysis of display system performance,” in Digital Images and Human Vision (A. B. Watson, ed.), pp. 163–178, Cambridge, MA: The MIT Press, 1993.

    Google Scholar 

  18. R. J. Safranek and J. D. Johnston, “A perceptually tuned sub-band image coder with image dependent quantization and post-quantization data compression,” in Proc. ICASSP-89, vol. 3, (Glasgow, Scotland), pp. 1945–1948, May 1989.

    Google Scholar 

  19. A. B.Watson, “DCTune: a technique for visual optimization of dct quantization matrices for individual images,” Society for Information Display Digest of Technical Papers, vol. 24, pp. 946–949, 1993.

    Google Scholar 

  20. K. Seshadrinathan, R. J. Safranek, J. Chen, T. N. Pappas, H. R. Sheikh, E. P. Simoncelli, Z. Wang and A. C. Bovik. Image quality assessment. In A. C. Bovik, editor, The Essential Guide to Image Processing, chapter 20. Academic Press, 2009.

    Google Scholar 

  21. C. J. van den Branden Lambrecht and O. Verscheure, “Perceptual quality measure using a spatiotemporal model of the human visual system,” in Proc. SPIE, vol. 2668, no. 1. San Jose, CA, USA: SPIE, Mar. 1996, pp. 450–461.

    Google Scholar 

  22. S. Winkler, “Perceptual distortion metric for digital color video,” Proc. SPIE, vol. 3644, no. 1, pp. 175–184, May 1999.

    Article  Google Scholar 

  23. E. P. Simoncelli, W. T. Freeman, E. H. Adelson, and D. J. Heeger, “Shiftable multiscale transforms,” IEEE Trans. Inform. Theory, vol. 38, pp. 587–607, Mar. 1992.

    Article  MathSciNet  Google Scholar 

  24. A. B. Watson, J. Hu, and J. F. McGowan III, “Digital video quality metric based on human vision,” J. Electron. Imaging, vol. 10, no. 1, pp. 20–29, Jan. 2001.

    Article  Google Scholar 

  25. M. Masry, S. S. Hemami, and Y. Sermadevi, “A scalable wavelet-based video distortion metric and applications,”Circuits and Systems for Video Technology, IEEE Transactions on, vol. 16, no. 2, pp. 260–273, 2006.

    Article  Google Scholar 

  26. H. Peterson, A.J. Ahumada, Jr. and A. Watson, ”An Improved Detection Model for DCT Coefficient Quantization,” Human Vision and Electronic Imaging, Proc. SPIE, 1913, 191–201

    Google Scholar 

  27. M. Carnec, P. Le Callet, and D. Barba, “Objective quality assessment of color images based on a generic perceptual reduced reference,” Signal Processing: Image Communication, Volume 23 , Issue 4, Pages 239–256, April 2008.

    Article  Google Scholar 

  28. K. Seshadrinathan and A. C. Bovik. Video quality assessment. In A. C. Bovik, editor, The Essential Guide to Video Processing, chapter 14. Academic Press, 2009.

    Google Scholar 

  29. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process, vol. 13, no. 4, pp. 600–612, 2004.

    Article  Google Scholar 

  30. Z. Wang and A. C. Bovik, “A universal image quality index,” IEEE Signal Processing Letters, vol. 9, no. 3, pp. 81–84, 2002.

    Article  Google Scholar 

  31. Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for image quality assessment,” in Thirty-Seventh Asilomar Conf. on Signals, Systems and Computers, Pacific Grove, CA, 2003.

    Google Scholar 

  32. Z. Wang and E. P. Simoncelli, “Translation insensitive image similarity in complex wavelet domain,” in IEEE Intl. Conf. Acoustics, Speech, and Signal Process., Philadelphia, PA, 2005.

    Google Scholar 

  33. M. P. Sampat, Z. Wang, S. Gupta, A. C. Bovik and M. K. Markey, ”Complex wavelet structural similarity: A new image similarity index,” IEEE Transactions on Image Processing, to appear 2009.

    Google Scholar 

  34. Z. Wang and X. Shang, “Spatial pooling strategies for perceptual image quality assessment,” in IEEE International Conference on Image Processing, Jan. 1996

    Google Scholar 

  35. A. K. Moorthy and A. C. Bovik, “Visual importance pooling for image quality assessment,” IEEE Journal of Selected Topics in Signal Processing, Special Issue on Visual Media Quality Assessment, to appear, April 2009.

    Google Scholar 

  36. Z. Wang, L. Lu, and A. C. Bovik, “Video quality assessment based on structural distortion measurement,” Signal Processing: Image Communication, vol. 19, no. 2, pp. 121–132, Feb. 2004.

    Article  Google Scholar 

  37. A. Srivastava, A. B. Lee, E. P. Simoncelli, and S.-C. Zhu, “On advances in statistical modeling of natural images,” J. Math. Imag. Vis., vol. 18, pp. 17–33, 2003.

    Article  MATH  MathSciNet  Google Scholar 

  38. E. P. Simoncelli and B. A. Olshausen, “Natural image statistics and neural representation,” Annu. Rev. Neurosci., vol. 24, pp. 1193–1216, May 2001.

    Article  Google Scholar 

  39. H. R. Sheikh and A. C. Bovik, “A visual information fidelity approach to video quality assessment,” First International Workshop on Video Processing and Quality Metrics for Conusmer Electronics, Jan. 2005.

    Google Scholar 

  40. H. R. Sheikh and A. C. Bovik, “Image information and visual quality,” IEEE Trans. Image Process, vol. 15, no. 2, pp. 430–444, 2006.

    Article  Google Scholar 

  41. H. R. Sheikh, A. C. Bovik, and G. de Veciana, “An information fidelity criterion for image quality assessment using natural scene statistics,” IEEE Trans. Image Process., vol. 14, no. 12, pp. 2117–2128, 2005.

    Article  Google Scholar 

  42. J. Malo, I. Epifanio, R. Navarro, and E. P. Simoncelli, “Non-linear image representation for efficient perceptual coding”, IEEE Transactions on Image Processing, vol.15(1), pp. 68–80, Jan 2006.

    Article  Google Scholar 

  43. J. Portilla and E. P. Simoncelli, “ A parametric texture model based on joint statistics of complex wavelet coefficients”, International Journal of Computer Vision, vol.40(1), pp. 49–71, Dec 2000.

    Article  MATH  Google Scholar 

  44. J. A. Guerrero-Colón, E. P. Simoncelli , and J. Portilla, “Image denoising using mixtures of Gaussian scale mixtures “, IEEE International Conference on Image Processing, pp. 565–568, Oct 2008.

    Google Scholar 

  45. M. J. Wainwright, E. P. Simoncelli, and A. S. Wilsky, “Random cascades on wavelet trees and their use in analyzing and modeling natural images,” Applied and Computational Harmonic Analysis, vol. 11, pp. 89–123, 2001.

    Article  MATH  MathSciNet  Google Scholar 

  46. M. J. Wainwright and E. P. Simoncelli, “Scale Mixtures of Gaussians and the statistics of natural images”, Adv. Neural Information Processing Systems (NIPS’99), vol.12 pp. 855–861, May 2000.

    Google Scholar 

  47. A. P. Hekstra, J. G. Beerends, D. Ledermann, F. E. de Caluwe, S. Kohler, R. H. Koenen, S. Rihs, M. Ehrsam, and D. Schlauss, “PVQM - A perceptual video quality measure,” Signal Proc.: Image Comm. vol. 17, pp. 781–798, 2002.

    Article  Google Scholar 

  48. Opticom. [Online]. Available: http://www.opticom.de/technology/pevq-video-quality-testing.html

  49. M. Malkowski and D. Claben, “Performance of video telephony services in UMTS using live measurements and network emulation,” Wireless Personal Comm., vol. 1, pp. 19–32, 2008.

    Article  Google Scholar 

  50. M. Barkowsky, J. Bialkowski, R. Bitto, and A. Kaup, “Temporal registration using 3D phase correlation and a maximum likelihood approach in the perceptual evaluation of video quality,” in IEEE Workshop on Multimedia Signal Proc., 2007.

    Google Scholar 

  51. The Video Quality Experts Group. (2000) Final report from the video quality experts group on the validation of objective quality metrics for video quality assessment. [Online]. Available: http://www.its.bldrdoc.gov/vqeg/projects/frtv phaseI

  52. Objective perceptual multimedia video quality measurement in the presence of a full reference, International Telecommunications Union Std. ITU-T Rec. J. 247, 2008.

    Google Scholar 

  53. M. H. Pinson and S. Wolf, “A new standardized method for objectively measuring video quality,” IEEE Trans. Broadcast., vol. 50, no. 3, pp. 312–322, Sep. 2004.

    Article  Google Scholar 

  54. The Video Quality Experts Group. (2003) Final VQEG report on the validation of objective models of video quality assessment. [Online]. Available: http://www.ts.bldrdoc.gov/vqeg/projects/frtv phaseII

  55. Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference, International Telecommunications Union Std. ITU-T Rec. J. 144, 2004.

    Google Scholar 

  56. “Video quality metric.” [Online]. Available: http://www.its.bldrdoc.gov/n3/video/VQM_ software.php

  57. M. Yuen and H. R. Wu, “A survey of hybrid MC/DPCM/DCT video coding distortions,” Signal Processing, vol. 70, no. 3, pp. 247–278, Nov. 1998.

    Article  MATH  Google Scholar 

  58. J. A. Movshon and W. T. Newsome, “Visual response properties of striate cortical neurons projecting to Area MT in macaque monkeys,” J. Neurosci., vol. 16, no. 23, pp. 7733–7741, 1996.

    Google Scholar 

  59. Z.Wang and Q. Li, “Video quality assessment using a statistical model of human visual speed perception.” J Opt Soc Am A Opt Image Sci Vis, vol. 24, no. 12, pp. B61–B69, Dec 2007.

    Article  Google Scholar 

  60. A. A. Stocker and E. P. Simoncelli, “Noise characteristics and prior expectations in human visual speed perception,” Nature Neuroscience, 9, 578–585 (2006).

    Article  Google Scholar 

  61. Black, M. J. and Anandan, P., “The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields,” Computer Vision and Image Understanding, 63, 75–104 (1996).

    Article  Google Scholar 

  62. K. Seshadrinathan and A. C. Bovik, “Spatio-temporal quality assessment of natural videos,” IEEE Transactions on Image Processing, submitted for publication.

    Google Scholar 

  63. K. Seshadrinathan and A. C. Bovik, “A structural similarity metric for video based on motion models,” IEEE International Conference on Acoustics, Speech, and Signal Processing, 2007.

    Google Scholar 

  64. D. J. Fleet and A. D. Jepson, “Computation of component image velocity from local phase information,” International Journal of Computer Vision, vol. 5, no. 1, pp. 77–104, 1990.

    Article  Google Scholar 

  65. D. J. Heeger, “Optical flow using spatiotemporal filters,” International Journal of Computer Vision, vol. 1, no. 4, pp. 279–302, 1987.

    Article  Google Scholar 

  66. E. H. Adelson and J. R. Bergen, “Spatiotemporal energy models for the perception of motion.” J Opt Soc Am A, vol. 2, no. 2, pp. 284–299, Feb 1985.

    Article  Google Scholar 

  67. N. J. Priebe, S. G. Lisberger, and J. A. Movshon, “Tuning for spatiotemporal frequency and speed in directionally selective neurons of macaque striate cortex.” J Neurosci, vol. 26, no. 11, pp. 2941–2950, Mar 2006.

    Article  Google Scholar 

  68. E. P. Simoncelli and D. J. Heeger, “A model of neuronal responses in visual area MT,” Vision Res, vol. 38, no. 5, pp. 743–761, Mar 1998.

    Article  Google Scholar 

  69. J. G. Daugman, “Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters,” Journal of the Optical Society of America A (Optics and Image Science), vol. 2, no. 7, pp. 1160–1169, 1985.

    Article  Google Scholar 

  70. P. C. Teo and D. J. Heeger, “Perceptual image distortion,” in Proceedings of the IEEE International Conference on Image Processing. IEEE, 1994, pp. 982–986 vol.2.

    Google Scholar 

  71. K. Seshadrinathan and A. C. Bovik, “Unifying analysis of full reference image quality assessment,” in IEEE Intl. Conf. on Image Proc., 2008.

    Google Scholar 

  72. A. B. Watson and J. Ahumada, A. J., “Model of human visual-motion sensing,” Journal of the Optical Society of America A (Optics and Image Science), vol. 2, no. 2, pp. 322–342, 1985.

    Article  Google Scholar 

  73. H. Frank and S. C. Althoen, “The coefficient of variation,” in Statistics: Concepts and Applications. Cambridge, Great Britan: Cambridge University Press., 1995, pp. 58–59.

    Google Scholar 

  74. K. Seshadrinathan, “Video quality assessment based on motion models,” Ph.D. dissertation, University of Texas at Austin, 2008.

    Google Scholar 

  75. H. R. Sheikh, M. F. Sabir, and A. C. Bovik, “A statistical evaluation of recent full reference image quality assessment algorithms,” IEEE Transactions on Image Processing, vol. 15, no. 11, pp. 3440–3451, Nov. 2006.

    Article  Google Scholar 

  76. LIVE image quality assessment database. [Online]. Available: http://live.ece.utexas.edu/research/quality/subjective.html

  77. Wang, Z. and Bovik, A. C., “Mean squared error: Love it or leave it? - a new look at fidelity measures.” IEEE Signal Processing Magazine. January 2009.

    Google Scholar 

  78. “Video coding for low bit rate communication”, ITU Recommendation H.263.

    Google Scholar 

  79. “Generic coding of moving pictures and associated audio information - part 2: Video,” 1994, ITU-T and ISO/IEC JTC 1. ITU-T Recommendation H.262 and ISO/IEC 13 818–2 (MPEG-2).

    Google Scholar 

  80. “Advanced video coding,” 2003, ISO/IEC 14496–10 and ITU-T Rec. H.264.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Texas, USA

    Anush K. Moorthy, Kalpana Seshadrinathan & Alan C. Bovik

Authors
  1. Anush K. Moorthy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kalpana Seshadrinathan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alan C. Bovik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alan C. Bovik .

Editor information

Editors and Affiliations

  1. Dept. Computer Science &, Florida Atlantic University, Glades Road 777, Boca Raton, 33431, U.S.A.

    Borko Furht

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Moorthy, A.K., Seshadrinathan, K., Bovik, A.C. (2009). Digital Video Quality Assessment Algorithms. In: Furht, B. (eds) Handbook of Multimedia for Digital Entertainment and Arts. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-89024-1_6

Download citation

  • DOI: https://doi.org/10.1007/978-0-387-89024-1_6

  • Published:

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-0-387-89023-4

  • Online ISBN: 978-0-387-89024-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics