Tradeoffs in subjective testing methods for image and video quality assessment

David M. Rouse; Romuald Pépion; Patrick Le Callet; Sheila S. Hemami

doi:10.1117/12.845389

17 February 2010 Tradeoffs in subjective testing methods for image and video quality assessment

David M. Rouse, Romuald Pépion, Patrick Le Callet, Sheila S. Hemami

Proceedings Volume 7527, Human Vision and Electronic Imaging XV; 75270F (2010) https://doi.org/10.1117/12.845389
Event: IS&T/SPIE Electronic Imaging, 2010, San Jose, California, United States

Abstract

The merit of an objective quality estimator for either still images or video is gauged by its ability to accurately estimate the perceived quality scores of a collection of stimuli. Encounters with radically different distortion types that arise in novel media representations require that researchers collect perceived quality scores representative of these new distortions to confidently evaluate a candidate objective quality estimator. Two common methods used to collect perceived quality scores are absolute categorical rating (ACR)¹ and subjective assessment for video quality (SAMVIQ).^{2, 3} The choice of a particular test method affects the accuracy and reliability of the data collected. An awareness of the potential benefits and/or costs attributed to the ACR and SAMVIQ test methods can guide researchers to choose the more suitable method for a particular application. This paper investigates the tradeoffs of these two subjective testing methods using three different subjective databases that have scores corresponding to each method. The subjective databases contain either still-images or video sequences. This paper has the following organization: Section 2 summarizes the two test methods compared in this paper, ACR and SAMVIQ. Section 3 summarizes the content of the three subjective databases used to evaluate the two test methods. An analysis of the ACR and SAMVIQ test methods is presented in Section 4. Section 5 concludes this paper.

Citation Download Citation

David M. Rouse, Romuald Pépion, Patrick Le Callet, and Sheila S. Hemami "Tradeoffs in subjective testing methods for image and video quality assessment", Proc. SPIE 7527, Human Vision and Electronic Imaging XV, 75270F (17 February 2010); https://doi.org/10.1117/12.845389

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available