skip to main content
research-article

Video quality assessment for computer graphics applications

Published: 15 December 2010 Publication History

Abstract

Numerous current Computer Graphics methods produce video sequences as their outcome. The merit of these methods is often judged by assessing the quality of a set of results through lengthy user studies. We present a full-reference video quality metric geared specifically towards the requirements of Computer Graphics applications as a faster computational alternative to subjective evaluation. Our metric can compare a video pair with arbitrary dynamic ranges, and comprises a human visual system model for a wide range of luminance levels, that predicts distortion visibility through models of luminance adaptation, spatiotemporal contrast sensitivity and visual masking. We present applications of the proposed metric to quality prediction of HDR video compression and temporal tone mapping, comparison of different rendering approaches and qualities, and assessing the impact of variable frame rate to perceived quality.

Supplementary Material

Supplemental material. (161-165-0190-auxiliary.zip)
Contents of the Auxilary Files Archive: modelfest_calibration_results.png: The metric response to the Modelfest stimuli set., video_metric_supplement.pdf: Formulas omitted for brevity in the main publication. video_metric_validation.pdf: Detailed discussion of the validation procedure. video_metric_video.mp4: A video sequence summarizing the main contribultions and application areas of the metric.

References

[1]
Aydin, T. O., Mantiuk, R., Myszkowski, K., and Seidel, H.-P. 2008. Dynamic range independent image quality assessment. In Proc. of ACM SIGGRAPH, vol. 27(3). Article 69.
[2]
Bavoil, L., Sainz, M., and Dimitrov, R. 2008. Image-space horizon-based ambient occlusion. In SIGGRAPH '08: ACM SIGGRAPH 2008 talks, ACM, New York, NY, USA, 1--1.
[3]
Bittner, J., Wimmer, M., Piringer, H., and Purgathofer, W. 2004. Coherent hierarchical culling: Hardware occlusion queries made useful. Computer Graphics Forum 23, 3 (Sept.), 615--624. Proceedings EUROGRAPHICS 2004.
[4]
Bolin, M., and Meyer, G. 1998. A perceptually based adaptive sampling algorithm. In Proc. of Siggraph '98, 299--310.
[5]
Dachsbacher, C., and Stamminger, M. 2005. Reflective shadow maps. In I3D '05: Proceedings of the 2005 symposium on Interactive 3D graphics and games, ACM, New York, NY, USA, 203--231.
[6]
Daly, S. 1993. The Visible Differences Predictor: An algorithm for the assessment of image fidelity. In Digital Images and Human Vision, MIT Press, A. B. Watson, Ed., 179--206.
[7]
Daly, S. J. 1998. Engineering observations from spatiovelocity and spatiotemporal visual models. SPIE, B. E. Rogowitz and T. N. Pappas, Eds., vol. 3299, 180--191.
[8]
Drago, F., Myszkowski, K., Annen, T., and N. Chiba. 2003. Adaptive logarithmic mapping for displaying high contrast scenes. Computer Graphics Forum 22, 3.
[9]
Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. In SIGGRAPH '02: Proceedings of the 29th annual conference on Computer graphics and interactive techniques, ACM Press, 249--256.
[10]
Ferwerda, J., and Pellacini, F. 2003. Functional difference predictors (fdps): measuring meaningful image differences. In Signals, Systems and Computers, 2003. Conference Record of the Thirty-Seventh Asilomar Conference on, vol. 2, 1388--1392 Vol.2.
[11]
Fredericksen, R. E., H. R. F. 1998. Estimating multiple temporal mechanisms in human vision. In Vision Research, vol. 38, 1023--1040.
[12]
Freeman, W. T., and Adelson, E. H. 1991. The design and use of steerable filters. Pattern Analysis and Machine Intelligence, IEEE Transactions on 13, 9, 891--906.
[13]
Herzog, R., Eisemann, E., Myszkowski, K., and Seidel, H.-P. 2010. Spatio-temporal upsampling on the GPU. In I3D '10: Proceedings of the 2010 symposium on Interactive 3D graphics and games, ACM, New York, NY, USA, 91--98.
[14]
ITU-T. 1999. Subjective video quality assessment methods for multimedia applications.
[15]
Lindh, P., and van den Branden Lambrecht, C. 1996. Efficient spatio-temporal decomposition for perceptual processing of video sequences. In Proceedings of International Conference on Image Processing ICIP'96, IEEE, vol. 3 of Proc. of IEEE, 331--334.
[16]
Lubin, J. 1995. Vision Models for Target Detection and Recognition. World Scientific, ch. A Visual Discrimination Model for Imaging System Design and Evaluation, 245--283.
[17]
Lukin, A. 2009. Improved visible differences predictor using a complex cortex transform. GraphiCon, 145--150.
[18]
Mantiuk, R., Krawczyk, G., Myszkowski, K., and Seidel, H.-P. 2004. Perception-motivated high dynamic range video encoding. ACM Trans. Graph. 23, 3, 733--741.
[19]
Mantiuk, R., Daly, S., Myszkowski, K., and Seidel, H.-P. 2005. Predicting visible differences in high dynamic range images - model and its calibration. In Human Vision and Electronic Imaging X, vol. 5666 of SPIE Proceedings Series, 204--214.
[20]
Masry, M. A., and Hemami, S. S. 2004. A metric for continuous quality evaluation of compressed video with severe distortions. Signal Processing: Image Communication 19, 2, 133--146.
[21]
Myszkowski, K., Rokita, P., and Tawara, T. 2000. Perception-based fast rendering and antialiasing of walkthrough sequences. IEEE Transactions on Visualization and Computer Graphics 6, 4, 360--379.
[22]
Myszkowski, K., Tawara, T., Akamine, H., and Seidel, H.-P. 2001. Perception-guided global illumination solution for animation rendering. In SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, ACM, New York, NY, USA, 221--230.
[23]
Pattanaik, S. N., Tumblin, J. E., Yee, H., and Greenberg, D. P. 2000. Time-dependent visual adaptation for fast realistic image display. In Proc. of ACM SIGGRAPH 2000, 47--54.
[24]
Reeves, W. T., Salesin, D. H., and Cook, R. L. 1987. Rendering antialiased shadows with depth maps. In SIGGRAPH '87: Proceedings of the 14th annual conference on Computer graphics and interactive techniques, ACM, New York, NY, USA, 283--291.
[25]
Ritschel, T., Grosch, T., and Seidel, H.-P. 2009. Approximating dynamic global illumination in image space. In I3D '09: Proceedings of the 2009 symposium on Interactive 3D graphics and games, ACM, New York, NY, USA, 75--82.
[26]
Rushmeier, H., Ward, G., Piatko, C., Sanders, P., and Rust, B. 1995. Comparing real and synthetic images: some ideas about metrics. In Rendering Techniques '95, Springer, P. Hanrahan and W. Purgathofer, Eds., 82--91.
[27]
Sampat, M. P., Wang, Z., Gupta, S., Bovik, A. C., and Markey, M. K. 2009. Complex wavelet structural similarity: A new image similarity index. Image Processing, IEEE Transactions on 18, 11 (Nov.), 2385--2401.
[28]
Schwarz, M., and Stamminger, M. 2009. On predicting visual popping in dynamic scenes. In APGV '09: Proceedings of the 6th Symposium on Applied Perception in Graphics and Visualization, ACM, New York, NY, USA, 93--100.
[29]
Seshadrinathan, K., and Bovik, A. 2007. A structural similarity metric for video based on motion models. In Acoustics, Speech and Signal Processing, 2007. ICASSP 2007. IEEE International Conference on, vol. 1, I-869--I-872.
[30]
Seshadrinathan, K., and Bovik, A. C. 2010. Motion tuned spatio-temporal quality assessment of natural videos. Image Processing, IEEE Transactions on 19, 2 (Feb.), 335--350.
[31]
van den Branden Lambrecht, C., and Verscheure, O. 1996. Perceptual Quality Measure using a Spatio-Temporal Model of the Human Visual System. In IS&T/SPIE.
[32]
van den Branden Lambrecht, C., Costantini, D., Sicuranza, G., and Kunt, M. 1999. Quality assessment of motion rendition in video coding. Circuits and Systems for Video Technology, IEEE Transactions on 9, 5 (Aug), 766--782.
[33]
Wandell, B. A. 1995. Foundations of Vision. Sinauer Associates, Inc.
[34]
Wang, Z., and Simoncelli, E. 2005. Translation insensitive image similarity in complex wavelet domain. In Acoustics, Speech, and Signal Processing, 2005. Proceedings. (ICASSP '05). IEEE International Conference on, vol. 2, 573--576.
[35]
Watson, A. B., and Malo, J. 2002. Video quality measures based on the standard spatial observer. In ICIP (3), 41--44.
[36]
Watson, A. B., Hu, J., and Iii, J. F. M. 2001. DVQ: A digital video quality metric based on human vision. Journal of Electronic Imaging 10, 20--29.
[37]
Watson, A. B. 1986. Temporal sensitivity. In Handbook of Perception and Human Performance, K. R. Boff, L. Kaufman, and J. P. Thomas, Eds. John Wiley and Sons, New York, 6-1-6-43.
[38]
Watson, A. 1987. The Cortex transform: rapid computation of simulated neural images. Comp. Vision Graphics and Image Processing 39, 311--327.
[39]
Winkler, S. 1999. A perceptual distortion metric for digital color video. In Proceedings of the SPIE Conference on Human Vision and Electronic Imaging, IEEE, vol. 3644 of Controlling Chaos and Bifurcations in Engineering Systems, 175--184.
[40]
Winkler, S. 2005. Digital Video Quality: Vision Models and Metrics. Wiley.
[41]
Yee, H., Pattanaik, S., and Greenberg, D. P. 2001. Spatiotemporal sensitivity and visual attention for efficient rendering of dynamic environments. ACM Trans. Graph. 20, 1, 39--65.

Cited By

View all
  • (2025)Luminance decomposition and reconstruction for high dynamic range Video Quality AssessmentPattern Recognition10.1016/j.patcog.2024.111011158(111011)Online publication date: Feb-2025
  • (2024)Spatial–Temporal Analysis-Based Video Quality Assessment: A Two-Stream Convolutional Network ApproachElectronics10.3390/electronics1310187413:10(1874)Online publication date: 10-May-2024
  • (2024)castleCSF — A contrast sensitivity function of color, area, spatiotemporal frequency, luminance and eccentricityJournal of Vision10.1167/jov.24.4.524:4(5)Online publication date: 4-Apr-2024
  • Show More Cited By

Index Terms

  1. Video quality assessment for computer graphics applications

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Graphics
      ACM Transactions on Graphics  Volume 29, Issue 6
      December 2010
      480 pages
      ISSN:0730-0301
      EISSN:1557-7368
      DOI:10.1145/1882261
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 15 December 2010
      Published in TOG Volume 29, Issue 6

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. high dynamic range video
      2. human visual perception
      3. subjective video quality assessment
      4. temporal artifacts
      5. video quality metrics

      Qualifiers

      • Research-article

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)36
      • Downloads (Last 6 weeks)1
      Reflects downloads up to 13 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2025)Luminance decomposition and reconstruction for high dynamic range Video Quality AssessmentPattern Recognition10.1016/j.patcog.2024.111011158(111011)Online publication date: Feb-2025
      • (2024)Spatial–Temporal Analysis-Based Video Quality Assessment: A Two-Stream Convolutional Network ApproachElectronics10.3390/electronics1310187413:10(1874)Online publication date: 10-May-2024
      • (2024)castleCSF — A contrast sensitivity function of color, area, spatiotemporal frequency, luminance and eccentricityJournal of Vision10.1167/jov.24.4.524:4(5)Online publication date: 4-Apr-2024
      • (2024)DashReStreamer: Framework for Creation of Impaired Video Clips under Realistic Network ConditionsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364001621:1(1-26)Online publication date: 16-Dec-2024
      • (2024)MNSS: Neural Supersampling Framework for Real-Time Rendering on Mobile DevicesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.325914130:7(4271-4284)Online publication date: 1-Jul-2024
      • (2022)Nonlinear Compliant Modes for Large-deformation Analysis of Flexible StructuresACM Transactions on Graphics10.1145/356895242:2(1-11)Online publication date: 22-Nov-2022
      • (2022)Perceptual Visibility Model for Temporal Contrast Changes in PeripheryACM Transactions on Graphics10.1145/356424142:2(1-16)Online publication date: 22-Nov-2022
      • (2022)Planar Panels and Planar Supporting Beams in Architectural StructuresACM Transactions on Graphics10.1145/356105042:2(1-17)Online publication date: 22-Nov-2022
      • (2022)Interlocking Spiral Drawings Inspired by M. C. Escher’s Print WhirlpoolsACM Transactions on Graphics10.1145/356071142:2(1-17)Online publication date: 22-Nov-2022
      • (2022)PTRM: Perceived Terrain Realism MetricACM Transactions on Applied Perception10.1145/351424419:2(1-22)Online publication date: 11-Jul-2022
      • Show More Cited By

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media