James R. Lewis
Abstract
A three-item after-scenario questionnaire was used in three related usability tests in different areas of the United States. The studies had eight scenarios in common. After participants finished a scenario, they completed the After-Scenario Questionnaire (the ASQ). A factor analysis of the responses to the ASQ items revealed that an eight-factor solution explained 94 percent of the variability of the 24 (eight scenarios by three items per scenario) items. The varimax-rotated factor pattern showed that these eight factors were clearly associated with the eight scenarios. The benefit of this research to system designers is that this three-item questionnaire has acceptable psychometric properties of reliability, sensitivity, and concurrent validity, and may be used with confidence in other, similar usability studies.
- Chin, J. P., Diehl, V. A., and Norman, K. (1988). Development of an instrument measuring user satisfaction of the human-computer interface. In Proceedings of the CHI'88 Conference on Human Factors in Computing Systems (pp. 213--218). New York, NY: Association for Computing Machinery. Google Scholar
Digital Library
- Cliff, N. (1987). Analyzing multivariate data. San Diego, CA: Harcourt Brace Jovanovich.Google Scholar
- Ives, B., Olson, M. H., and Baroudi, J. J. (1983). The measurement of user satisfaction. Communications of the ACM, 26, 785--793. Google ScholarDigital Library
- Kirakowski, J. and Corbett, M. (1988). The Computer User Satisfaction Inventory (CUSI); Manual and scoring key. Ireland: University College of Cork, Human Factors Research Group.Google Scholar
- Lewis, J. R. (1990). A psychometric evaluation of a post-study system usability questionnaire: The PSSUQ (IBM Tech. Report 54. 535). Boca Raton, FL: IBM Corp.Google Scholar
- McIver, J. P. and Carmines, E. G. (1981). Unidimensional scaling. Sage University Paper series on Quantitative Applications in the Social Sciences, series no. 07--024. Beverly Hills, CA: Sage Publications.Google Scholar
- Nunnally, J. C. (1978). Psychometric theory. New York, NY: McGraw-Hill.Google Scholar
- SAS Institute. (1985). SAS user's guide: statistics, version 5 edition. Cary, NC: SAS Institute.Google Scholar
Index Terms
- Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ
Recommendations
AN AFTER-SCENARIO QUESTIONNAIRE FOR USABILITY STUDIES: PSYCHOMETRIC EVALUATION OVER THREE TRIALS
The purpose of this report is to (1) describe a psychometric evaluation of a printer scenario questionnaire (PSQ) that was used to assess user satisfaction during scenario-based usability studies of printers and (2) compare the psychometric properties ...
Improving Usability Evaluation by Automating a Standardized Usability Questionnaire
Design, User Experience, and Usability: Theory and PracticeAbstractUsability is one of the factors that determines the success of a software system. Aiming to improve it’s usability, it is necessary to evaluate the interfaces using scientific evaluation methods, like questionnaires. Often the results of these ...
Comments