Reliability, validity, and sensitivity of a single-item measure of online store usability
Research Highlights
►We examined the quality of a new single-item measure for online store usability. ►Results revealed an adequate reliability and convergent validity of the measure. ►Moreover good predictive validity and sensitivity were demonstrated. ►We conclude that the single-item measure of usability is a useful instrument.
Introduction
Besides usability, several factors, including trust and aesthetics, determine the success of an e-commerce interface (Schlosser et al., 2006, Hall and Hanna, 2004). As a result, the number of constructs and variables (e.g., user characteristics), which has to be considered in the evaluation of online stores, tends to be high. However, the number of items that can be administered in surveys is usually limited due to time and also space constraints (Braithwaite and Scott, 1991). The possibilities for integrating a large number of items are strongly limited, particularly, in online studies (Evans and Mathur, 2005). Consequently, it is necessary to include scales consisting of only a few items per construct, while still ensuring reliable and valid measurement.
According to the recommendations of numerous scholars (e.g., Churchill, 1979, Nunnally, 1978), researchers should preferably use multiple-item measures in which a latent construct is represented by a number of manifest items. This approach is in line with the tradition of psychometrics that considers the measurement of psychological constructs as traits, states, or abilities (Kline, 1979). Empirical studies that do not include multiple-item measures are less likely to be accepted for publication in scientific journals (Bergkvist and Rossiter, 2007). In contrast, practitioners often prefer single items to multiple-item measures, as there is less effort involved and costs are reduced. A single-item measure consists of one single item for the assessment of a latent construct. In fact, researchers have successfully applied single-item measures in a broad range of different research areas, such as clinical psychology (McKenzie and Marks, 1999), quality of life research (Cunny and Perri, 1991, Hyland and Sodergran, 1996), consumer research (Bergkvist and Rossiter, 2009, Sarstedt and Wilczynski, 2009), social and personality psychology (Aron et al., 1992), and occupational psychology (Dolbier et al., 2005, Scarpello and Campbell, 1983, Stanton et al., 2002a, Wanous and Hudy, 2001). The aim of this study is to explore the reliability, content validity, predictive validity, and the sensitivity of a single-item measure for usability of online stores.
Section snippets
The assessment of online store usability
According to the norm DIN EN ISO 9241-11 (1998), usability is defined as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”. Usability has to be considered as one of the main decisive factors in e-commerce interface success (Konradt et al., 2003, Weathers and Makienko, 2006). Consequently, usability experts emphasize that investment in improving this dimension of interface
Research questions and hypotheses
The purpose of our study is to explore the reliability, content validity, predictive validity, and the sensitivity of a single-item measure of online stores' usability based on a subjective customer's appraisal. Consistent with studies on the psychometric criteria of single-item measures (e.g., Dolbier et al., 2005, Oshagbemi, 1999, Wanous et al., 1997), as well as research on the success factors involved in e-commerce (e.g., Lee and Lee, 2004), we propose the following hypotheses. Firstly, we
Sample
378 participants completed the questionnaire in the context of an experimental study. Ten cases were excluded due to extensive missing data (more than 20%), resulting in a sample size of 368. A comparison between the full sample and the reduced sample showed no discrepancies concerning the distributions, means, and standard deviations of the assessed variables. The final sample included 47.6% females and 52.4% males with an average age of 32.7 years (SD=12.3; Min=15; Max=72). A considerably
Results
Since all participants evaluated two stores, the number of cases for data analysis was doubled (N=736).7
Discussion
The aim of this study was to examine the reliability, content validity, predictive validity, and sensitivity of a single-item measure for the usability of online stores. These findings are consistent with the results of previous studies conducted in other research contexts that focus on single-item measures (Dolbier et al., 2005, Scarpello and Campbell, 1983, Stanton et al., 2002a). In this study, we followed two different methods for determining the reliability of the single-item. Both factor
Acknowledgements
Research by the first author was supported by the German Research Foundation (Graduate School “Loosely Coupled Systems”). Parts of this study were presented at the Fourth International Conference on Cybernetics and Information Technologies, Systems and Applications (CITSA) 2007, July 12–15, 2007, in Orlando, Florida, USA. We thank Diana Hansen, Lea Schurawitzki, and Oliver Harm for their contributions to the data collection. We also gratefully acknowledge the helpful comments of Associate
References (101)
- et al.
Applications of structural equation modeling in marketing and consumer research: a review
International Journal of Research in Marketing
(1996) - et al.
Values
Current practice in measuring usability: challenges to usability studies and research
International Journal of Human–Computer Studies
(2006)- et al.
Predicting user satisfaction, strain and system usage of employee self-services
International Journal of Human–Computer Studies
(2006) - et al.
What is this evasive beast we call user satisfaction?
Interacting with Computers
(2003) - et al.
Experiential value: conceptualization, measurement and application in the catalog and Internet shopping environment
Journal of Retailing
(2001) - et al.
Critical factors for the aesthetic fidelity of web pages: empirical studies with professional web designers and users
Interacting with Computers
(2004) - et al.
The mechanics of trust: a framework for research and design
International Journal of Human–Computer Studies
(2005) The C-OAR-SE procedure for scale development in marketing
International Journal of Research in Marketing
(2002)- et al.
Customer satisfaction and loyalty in online and offline environments
International Journal of Research in Marketing
(2003)
What is beautiful is usable
Interacting with Computers
Task structure and the apparent duration of hierarchical search
International Journal of Human-Computer Studies
Factors influencing the usage of websites: the case of a generic portal in the Netherlands
Information & Management
Assessing the relationships between e-tail success and product and web site factors
Journal of Interactive Marketing
Structural equation modeling in practice: a review and recommended two-step approach
Psychological Bulletin
Designing samples
Inclusion of other in self-scale and the structure of interpersonal closeness
Journal of Personality and Social Psychology
Tailor-made single-item measures of doubly concrete constructs
International Journal of Advertising
The predictive validity of multiple-item versus single-item measures of the same constructs
Journal of Marketing Research
Cost Justifying Usability—An Update for the Internet Age
Handbook on Formative and Summative Evaluation of Student Learning
Translation and content analysis of oral and written materials
SUS—a quick and dirty usability scale
Vertrauen, Risiko und Usability bei der Nutzung von Internet-Apotheken (Trust, risk and usability of online pharmacies)
Reliability and Validity Assessment
Human–computer interaction: psychology as a science of design
International Journal of Human–Computer Studies
Determining causes and severity of end-user frustration
International Journal of Human–Computer Interaction
Development of an instrument measuring user satisfaction of the human–computer interface.
A paradigm for developing better measures of marketing constructs
Journal of Marketing Research
Constructing validity: basic issues in objective scale development
Psychological Assessment
Single-item vs. multiple-item measures of health-related quality of life
Psychological Reports
Index construction with formative indicators: an alternative to scale development
Journal of Marketing Research
What is beautiful is good
Journal of Personality and Social Psychology
Pre-purchase online information seeking. Search versus browse
Journal of Electronic Commerce Research
Reliability and validity of a single-item measure of job satisfaction
American Journal of Health Promotion
Do we really need multiple-item measures in service research?
Journal of Service Research
User-based evaluations
Affective design of e-commerce user interfaces: how to maximise perceived trustworthiness
When reputation engenders trust: an empirical investigation in business-to-consumer electronic commerce
Electronic Markets
The value of online surveys
Internet Research: Electronic Networking Applications and Policy
Using single-item measures for construct measurement in management research
Die Betriebswirtschaft
Single-item versus multiple-item measurement scales: an empirical comparison
Educational and Psychological Measurement
The IsoMetrics Usability Inventory: an operationalization of ISO 9241-10 supporting summative and formative evaluation of software systems
Behaviour & Information Technology
The relative importance of perceived ease of use in IS adoption: a study of e-commerce adoption
Journal of the Association for Information Systems
Methods for handling missing data
The impact of web page text-background colour combinations on readability, retention, aesthetics and behavioural intention
Behaviour & Information Technology
The effect of web site perceptions on initial trust in the owner company
International Journal of Electronic Commerce
Cited by (71)
Social media attachment: Conceptualization and formative index construction
2022, Journal of Business ResearchA Large-Scale Test of the Reality Constraint and Ingroup Bias Accounts of Women's Support for Male Privilege
2024, Psychology of Women QuarterlyInvestigating the Influence of COVID-Related Worry on University Enrollment Intentions: An Application of the Reasoned Action Model
2024, Journal of College Student Retention: Research, Theory and Practice