Janni Nielsen
ABSTRACT
One of the basic usability testing techniques the HCI community draws on, and which stands out as unique, is thinking aloud. We introduce the many names, uses and modifications of the classical think aloud technique, and ask the rhetorical question: What do researchers think they get when they ask people to think aloud? We answer it by discussing the classical work of Ericsson and Simon(1984), in particular their distinction between vocalisation, verbalisation and retrospective reports and the relation to short term memory. Reintroducing the psychological perspective and the focus on higher order cognitive processes, we argue that access to subjective experience is possible in terms of introspection and describe a technique that invites the user to become a participant in the analysis of his or her own cognitive processes. We suggest that use of think aloud has as a prerequisite explicit descriptions of design, test procedure and framework for analysis. We point out, however, that if the aim is to get access to human thinking, HCI research may benefit from experimental research.
- Bannon, L. (1991), From Human Factors to Human Actors: The Role of Psychology and Human-Computer Interaction Studies in Systems Design, in Design at work: Cooperative Design of Computer Systems., eds. Greenbaum, J. & Kyng, M., Hillsdale: Lawrence Erlbaum Associates, pp. 25-44. Google Scholar
- Bedny, G. & Meister, D. (1997), The Russian Theory of Activity: Current Applications to Design and Learning, Lawrence Erlbaum, Mahwah, New Yersey.Google Scholar
- Boren, M. T. & Ramey, J. (2000), Thinking aloud: Reconciling theory and practice, IEEE Transactions on Professional Communication, vol. 43, no. 3, pp. 261-278.Google Scholar
- Branch, J. L. (2000), Investigating the information-seeking processes of adolescents: The value of using think alouds and think afters, Library & Information Science Research, vol. 22, no. 4, pp. 371-392.Google ScholarCross Ref
- Bringham, B., John, R. & Lewis, C. (1991), Usability Testing of a Graphical Programming System: Things We Missed in a Programming Walkthrough, in Human factors in computing systems conference proceedings on Reaching through technology, New Orleans, Louisiana, United States, pp. 7-12. Google Scholar
- Buur, J. & Bagger, K. (1999), Replacing usability testing with user dialogue - How a Danish manufacturing company enhanced its product design process by supporting user participation, Communications of the ACM, vol. 42, no. 5, pp. 63-66. Google ScholarDigital Library
- Bødker, S. (1996), Applying AT to Video Analysis: How to make sense of Video Data in HCI, in Context and Consciousness, ed. Nardi, B. A., MIT Press, pp. 147-174.Google Scholar
- Clemmensen, T. (2002), Interest in theory and method in a Danish community of usability specialists (in Danish). Working paper, Copenhagen Business School, Department of Informatics, HCI research group., pp. 1-30.Google Scholar
- Dix, A., Finlay, J., Abowd, G. & Beale, R. (1997), Human-Computer Interaction, Prentice Hall. Google ScholarDigital Library
- Duncker, Karl(1945) On problem-solving, in Dashiell, John F.: Psychological Monographs, The American Psychological Association, Inc., Washington DC, vol. 58, pp. 1-114. (reprint, Westport, Conn. Greenwood, 1976).Google Scholar
- Ericsson, K. A. & Simon, H. A. (1984), Protocol Analysis. Verbal reports as data, Cambridge Massachusets.Google Scholar
- Ericsson, K. A. & Simon, H. A. (1998), How to study thinking in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking., Mind, Culture, & Activity, vol. 5, no. 3, pp. 178-186.Google ScholarCross Ref
- Flading, Janne(2002) Usability expert at B&O --- personal communicationGoogle Scholar
- Hackos, J. T. & Redish, J. C. (1998), User and Task Analysis for interface Design, Wiley, USA. Google ScholarDigital Library
- Kaptelinin, V. & Nardi, B. A. (1997), Activity Theory: Basic Concepts and Applications, Available: {http://www.acm.org/sigchi/chi97/proceedings/tutorial/bn.htm}. Google ScholarDigital Library
- Kari, K. & Arvonen, T. (1992), Identifying Potential CSCW Applications by Means of Activity Theory Concepts: A Case Example, in CSCW 92.Google Scholar
- Karsenty, L. (2001), Adapting verbal protocol methods to investigate speech systems use, Applied Ergonomics, vol. 32, no. 1, pp. 15-22.Google ScholarCross Ref
- Katalin, E. (2000), "Please, keep talking": The 'think-aloud' method in second language reading research, Novelty, vol. 7, no. 3.Google Scholar
- Kensing, F. (1998), Prompted Reflections: A Technique for Understanding complex Work, Interactions, vol. January-February 1998, pp. 7-15. Google ScholarDigital Library
- Kirkeby, O. F. (1988) Kroppens Tanke (The thought of the body), Samfundslitteratur, CopenhagenGoogle Scholar
- Koenemann-Belliveau, J., Carroll, J. M., Rosson, M. B. & Singley, M. K. (1994), Comparative usability evaluation: critical incidents and critical threads, in Human factors in computing systems: "Celebrating interdependence", ACM Press, New York, NY, USA, Boston, Massachusetts, United States, pp. 245-251. Google Scholar
- Kuutti, K. (1996), AT as a Potential Framework for Human-Computer Interaction Research., in Context and Consciousness., eds. Bødker, S. & Nardi, B. A., MIT Press, pp. 17-44. Google Scholar
- Mehlenbacher, B. (1993), Software usability: choosing appropriate methods for evaluating online systems and documentation, in Proceedings of the 11th annual international conference on Systems documentation, Waterloo, Ontario, Canada, pp. 209-222. Google ScholarDigital Library
- Molich, R. (1994), User friendly systems (in Danish), Teknisk forlag, København.Google Scholar
- Nielsen, Janni (1992), Designers' Decision Making: Design of a User Interface for a Picture SearchGoogle Scholar
- Nielsen, Janni & Christiansen, N. (2000), Mindtape: A Tool for Reflection in Participatory Design, in Participatory Design, New York, pp. 303-313. Database, Esprit 3066, AMODEUS RP7/WP14.Google Scholar
- Nielsen, J. (1994), Estimating the Number of Subjects Needed for a Thinking Aloud Test, International Journal of Human-Computer Studies, vol. 41, no. 3, pp. 385-397. Google ScholarDigital Library
- Nielsen, J. (1992). Evaluating the thinking aloud technique for use by computer scientists. In Hartson, H. R. and Hix, D. (Eds.): Advances in Human Google ScholarDigital Library
- Nisbett, R. E. & Wilson, T. D. (1977), Telling More Than We Can Know: Verbal Reports on Mental Processes, Psychological Review, vol. 84, no. 3, pp. 231-259.Google ScholarCross Ref
- Preece, J. (1994), Human-Computer Interaction, Addison-Wesley, England. Google ScholarDigital Library
- Preece, J., Rogers, Y. & Sharp, H. (2002), Interaction Design: Beyond Human-Computer Interaction, John Wiley & Sons. Google ScholarDigital Library
- Ritter, F. E. & Young, R. M. (2001), Embodied models as simulated users: introduction to this special issue on using cognitive models to improve interface design, International Journal of Human-Computer Studies, vol. 55, no. 1, pp. 1-14.Google ScholarDigital Library
- Rowley, D. E. (1994), Usability testing in the field: bringing the laboratory to the user., in Human factors in computing systems: "Celebrating interdependence". ACM Press, New York, NY, USA, Boston, Massachusetts, United States, pp. 252-257. Google Scholar
- Simpson, M. (1990), How usability testing can aid the development of online documentation, ACM SIGDOC Journal of Computer Documentation, vol. 14, no. 4. Google ScholarDigital Library
- Turner, P., Turner, S. & Horton, J. (1999), From description to requirements: an activity theoretic perspective, in Proceedings of the international ACM SIGGROUP conference on Supporting group work, ACM Press, New York, NY, USA, Phoenix, Arizona, United States, pp. 286-295. Google ScholarDigital Library
- Vermersch, P. (1999), Introspection as practice, in The View from Within - First person approaches to the study of consciousness, eds. Varela, F. & Shear, J., Imprint Academic, Exeter, UK, p. 320 pages. The chapter is also available from http://cognet.mit.edu/MITECS/Entry/voneche.html.Google Scholar
- Waes, L. v. (1998), Evaluating on-line and off-line searching behavior using thinking-aloud protocols to detect navigation barriers, in Proceedings of the sixteenth annual international conference on Computer documentation, ACM Press, New York, NY, USA, Quebec, Canada, pp. 180-183. Google ScholarDigital Library
- Wright, P. C. & Monk, A. F. (1990), The use of think-aloud evaluation methods in design, ACM SIGCHI Bulletin December 1990, vol. 23, no. 1. Google ScholarDigital Library
- Yeo, A. (1998), Cultural Effects in Usability Assessment, in CHI 98, Doctoral Consortium, pp. 74-76. Google ScholarDigital Library
Index Terms
- Getting access to what goes on in people's heads?: reflections on the think-aloud technique
Recommendations
Think-aloud protocols: a comparison of three think-aloud protocols for use in testing data-dissemination web sites for usability
CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsWe describe an empirical, between-subjects study on the use of think-aloud protocols in usability testing of a federal data-dissemination Web site. This double-blind study used three different types of think-aloud protocols: a traditional protocol, a ...
Think aloud: effects and validity
SIGDOC '12: Proceedings of the 30th ACM international conference on Design of communicationThink aloud is a commonly used usability method with roots in psychology. Although current think aloud practice was adapted from a standard method defined by Ericsson and Simon, there is no evidence of the use of a standard method by usability ...
Probing student problem solving skills in mathematical induction using a scenario based think aloud protocol
WCCCE '11: Proceedings of the 16th Western Canadian Conference on Computing EducationThink aloud protocol has traditionally been used to probe thinking and problem solving skills. In a series of student interview sessions where think aloud protocol was used to study student problem solving skills in mathematical induction at the ...
Comments