Skip to main content
SpringerLink
Log in

Reliability of Local Ground Truth Data for Image Quality Metric Assessment

  • Conference paper
  • First Online:
Book cover Image Processing and Communications Challenges 10 (IP&C 2018)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 892))

Included in the following conference series:

Abstract

Image Quality Metrics (IQMs) automatically detect differences between images. For example, they can be used to find aliasing artifact in the computer generated images. An obvious application is to test if the costly anti-aliasing techniques must be applied so that the aliasing is not visible to humans. The performance of IQMs must be tested based on the ground truth data, which is a set of maps that indicate the location of artifacts in the image. These maps are manually created by people during so called marking experiments. In this work, we evaluate two different techniques of marking. In the side-by-side experiment, people mark differences between two images displayed side-by-side on the screen. In the flickering experiment, images are displayed at the same location but are exchanged over time. We assess the performance of each technique and use the generated reference maps to evaluate the performance of the selected IQMs. The results reveal the better accuracy of the flickering technique.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Akenine-Möller, T., Haines, E., Hoffman, N.: Real-Time Rendering, 3rd edn. A K Peters Ltd., Wallesley (2008)

    Google Scholar 

  2. Čadík, M., Herzog, R., Mantiuk, R., Mantiuk, R., Myszkowski, K., Seidel, H.P.: Learning to predict localized distortions in rendered images. Comput. Graph. Forum 32(7), 401–410 (2013)

    Article  Google Scholar 

  3. Čadík, M., Herzog, R., Mantiuk, R., Myszkowski, K., Seidel, H.P.: New measurements reveal weaknesses of image quality metrics in evaluating graphics artifacts. ACM Trans. Graph. (TOG) 31(6), 147 (2012)

    Article  Google Scholar 

  4. Corsini, M., Larabi, M.C., Lavoué, G., Petřík, O., Váša, L., Wang, K.: Perceptual metrics for static and dynamic triangle meshes. Comput. Graph. Forum 32(1), 101–125 (2013)

    Article  Google Scholar 

  5. Lavoué, G., Mantiuk, R.: Quality assessment in computer graphics. In: Visual Signal Quality Assessment, pp. 243–286. Springer, Cham (2015)

    Google Scholar 

  6. Lissner, I., Preiss, J., Urban, P., Lichtenauer, M.S., Zolliker, P.: Image-difference prediction: from grayscale to color. IEEE Trans. Image Process. 22(2), 435–446 (2013)

    Article  MathSciNet  Google Scholar 

  7. Mantiuk, R., Kim, K.J., Rempel, A.G., Heidrich, W.: HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans. Graph. 30(4), 40:1–40:14 (2011)

    Google Scholar 

  8. Piórkowski, R., Mantiuk, R.: Using full reference image quality metrics to detect game engine artefacts. In: Proceedings of the ACM SIGGRAPH Symposium on Applied Perception, pp. 83–90. ACM (2015)

    Google Scholar 

  9. Piórkowski, R., Mantiuk, R., Siekawa, A.: Automatic detection of game engine artifacts using full reference image quality metrics. ACM Trans. Appl. Percept. (TAP) 14(3), 14 (2017)

    Google Scholar 

  10. Rushmeier, H.E., Rogowitz, B.E., Piatko, C.: Perceptual issues in substituting texture for geometry. In: Electronic Imaging, pp. 372–383. International Society for Optics and Photonics (2000)

    Google Scholar 

  11. Sergej, T., Mantiuk, R.: Perceptual evaluation of demosaicing artefacts. In: Image Analysis and Recognition. LNCS, vol. 8814, pp. 38–45. Springer, Cham (2014)

    Google Scholar 

  12. Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  13. Wang, Z., Bovik, A.: Modern Image Quality Assessment. Morgan & Claypool Publishers (2006)

    Google Scholar 

  14. Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: Conference Record of the Thirty-Seventh Asilomar Conference on Signals, Systems and Computers 2004, vol. 2, pp. 1398–1402. IEEE (2003)

    Google Scholar 

  15. Wolski, K., Giunchi, D., Ye, N., Didyk, P., Mantiuk, R., Seidel, H.P., Steed, A., Mantiuk, R.K.: Dataset and metrics for predicting local visible differences. ACM Trans. Graph. (2018)

    Google Scholar 

  16. Zhang, X., Wandell, B.A.: A spatial extension of cielab for digital color-image reproduction. Journal of the Society for Information Display 5(1), 61–63 (1997)

    Article  Google Scholar 

Download references

Acknowledgments

The project was partially funded by the Polish National Science Centre (decision number DEC-2013/09/B/ST6/02270).

Author information

Authors and Affiliations

  1. West Pomeranian University of Technology, Szczecin, Poland

    Rafał Piórkowski & Radosław Mantiuk

Authors
  1. Rafał Piórkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Radosław Mantiuk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rafał Piórkowski .

Editor information

Editors and Affiliations

  1. Institute of Telecommunications and Computer Science, University of Science and Technology, UTP, Bydgoszcz, Poland

    Michał Choraś

  2. Institute of Telecommunications and Computer Science, University of Science and Technology, UTP, Bydgoszcz, Poland

    Ryszard S. Choraś

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Piórkowski, R., Mantiuk, R. (2019). Reliability of Local Ground Truth Data for Image Quality Metric Assessment. In: Choraś, M., Choraś, R. (eds) Image Processing and Communications Challenges 10. IP&C 2018. Advances in Intelligent Systems and Computing, vol 892. Springer, Cham. https://doi.org/10.1007/978-3-030-03658-4_5

Download citation

Publish with us

Policies and ethics

Search

Navigation