skip to main content
10.1145/1297277.1297280acmotherconferencesArticle/Chapter ViewAbstractPublication PagesidcConference Proceedingsconference-collections
Article

A comparison of usability evaluation methods for child participants in a school setting

Published: 06 June 2007 Publication History

Abstract

This study assesses three usability evaluation methods (Active Intervention, Peer Tutoring and Cross-Age Tutoring) with children aged 6--8 years old within a school setting, using an interactive, educational multimedia product. Cross-Age Tutoring elicited significantly fewer comments than the other two methods, and 'plan' comments were significantly rarer than 'action' and 'perception and cognition' comments. In terms of the suitability of these evaluation methods for child participants, and context of use in this particular setting, Peer Tutoring appears to have the most potential. Usability evaluation methods need to better reflect children's motivations and the variety of contexts in which they use multimedia products. The implications for HCI practitioners are explored.

References

[1]
Bodker, S. and K. Madsen (1998). Methods and tools: context: an active choice in usability work. Interactions 5(4): 17--25.
[2]
Boren, M. and J. Ramey (2000). Thinking aloud; reconciling theory and practice. IEEE Transactions on Professional Communication 43(3): 261--278.
[3]
Branch, J. (2000). Investigating the information-seeking processes of adolescents: the value of using think alouds and think afters. Library and Information Science Research 22(4): 371--392.
[4]
Cole, M., and S. Cole, (1993). The Development of Children. Second edition, Scientific American Books, USA.
[5]
Cummings, R. (1985). Small group discussions and the microcomputer. Journal of Computer Assisted Learning 1:149--158.
[6]
Druin, A. (1999). Cooperative inquiry: developing new technologies for children with children. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems: the CHI is the Limit. CHI, USA. pp. 592--599.
[7]
Egloff, T. H. (2004). Edutainment: A Case Study of Interactive CD-ROM Playsets. ACM Computers in Entertainment 2(1): 1--22.
[8]
Ellis, S., and R. Siegler (1997). Planning and strategy choice, or why don't children plan when they should? In S. L. Friedman and E. K. Scholnick (Eds). Why, How, and When Do We Plan: The Developmental Psychology of Planning, Erlbaum, Hillsdale, NJ.
[9]
Ericsson, T., and H. Simon (1984). Protocol Analysis: Verbal Reports as Data. MIT Press, Cambridge, MA.
[10]
Eysenck, M. W., and M. T. Keane (1995). Cognitive Psychology: A Student's Handbook (Third Edition). Psychology Press, UK.
[11]
Flavell, J., P. Miller and S. Miller (2002). Cognitive Development. 4th Edition, Prentice Hall, NJ.
[12]
Gilutz, S., and J. Nielsen (2002). Usability of Websites for Children: 70 Design Guidelines. Nielsen Norman Group, http:www.nngroup.com/reports/kids. Site visited 5/5/04.
[13]
Halgren, S. L., T. Fernandes and D. Thomas (1995). Amazing animation: movie making for kids design briefing. In Proceedings of the SIGCHI conference on Human factors in computing systems. CHI, USA, pp. 519--525.
[14]
Hanna, L., K. Risden, and K. Alexander (1997). Guidelines for usability testing with children. Interactions 4(5): 9--14.
[15]
Hoysniemi, J., P. Hamalainen, and L. Turkki (2003). Using peer tutoring in evaluating usability. Interacting with Computers 15:203--225.
[16]
Karentsy, L. (2001). Adapting verbal protocol methods to investigate speech systems use. Applied Ergonomics 32(1): 15--22.
[17]
Kennedy, S. (1989). Using video in the BNR usability lab. SIGCHI Bulletin 92--95.
[18]
Markopoulos, P., and M. Bekker (2003). On the assessment of usability testing methods for children. Interacting with Computers 15:227--243.
[19]
Nielsen, J., T. Clemmensen and C. Yssing (2002). Getting access to what goes on in people's heads? -- Reflections on the think-aloud technique. In Proceedings of the second Nordic conference on Human-Computer Interaction. pp. 101--110.
[20]
Norman, D. (1986). Cognitive Engineering. In D. Norman and S. Draper (Eds), User Centred System Design, Lawrence Erlbaum Associates, Hillsdale, NJ. pp 31--61.
[21]
van Kesteren, I., M. Bekker, A. Vermeeren and P. Lloyd (2003). Assessing Usability Evaluation Methods On Their Effectiveness To Elicit Verbal Comments From Children Subjects. In Proceedings of the 2003 conference on Interaction design and children pp. 41--49.
[22]
Wellman, H., and J. Lempers (1977). The naturalistic communication abilities of two-year olds. Child Development 48:1052--1057.
[23]
Wixon, D. (2003). Evaluating Usability Methods: Why the Current Literature Fails the Practitioner. Interactions July and August: 29--34.

Cited By

View all
  • (2020)18 Years of ethics in child-computer interaction researchProceedings of the Interaction Design and Children Conference10.1145/3392063.3394407(161-183)Online publication date: 21-Jun-2020
  • (2020)Designing the Future of Children’s Mental Health ServicesAdministration and Policy in Mental Health and Mental Health Services Research10.1007/s10488-020-01038-x47:5(735-751)Online publication date: 6-Apr-2020
  • (2019)Usability Testing and Feedback Collection in a School Context: Case Poetry MachineErgonomics in Design: The Quarterly of Human Factors Applications10.1177/106480461878738227:3(17-23)Online publication date: 28-Jan-2019
  • Show More Cited By

Index Terms

  1. A comparison of usability evaluation methods for child participants in a school setting

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    IDC '07: Proceedings of the 6th international conference on Interaction design and children
    June 2007
    222 pages
    ISBN:9781595937476
    DOI:10.1145/1297277
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    • DOF: Det Obelske Familiefond
    • LEGO

    In-Cooperation

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 06 June 2007

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. active intervention
    2. child participants
    3. cross-age tutoring
    4. peer tutoring
    5. school setting
    6. usability evaluation

    Qualifiers

    • Article

    Conference

    IDC07
    Sponsor:
    • DOF

    Acceptance Rates

    Overall Acceptance Rate 172 of 578 submissions, 30%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)11
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 15 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2020)18 Years of ethics in child-computer interaction researchProceedings of the Interaction Design and Children Conference10.1145/3392063.3394407(161-183)Online publication date: 21-Jun-2020
    • (2020)Designing the Future of Children’s Mental Health ServicesAdministration and Policy in Mental Health and Mental Health Services Research10.1007/s10488-020-01038-x47:5(735-751)Online publication date: 6-Apr-2020
    • (2019)Usability Testing and Feedback Collection in a School Context: Case Poetry MachineErgonomics in Design: The Quarterly of Human Factors Applications10.1177/106480461878738227:3(17-23)Online publication date: 28-Jan-2019
    • (2018)A systematic review of usability quality attributes for the evaluation of mobile learning applications for children10.1063/1.5055494(020092)Online publication date: 2018
    • (2016)SmallTalkProceedings of the TEI '16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/2839462.2839481(253-261)Online publication date: 14-Feb-2016
    • (2016)The child's perspective as a guiding principleJournal of Biomedical Informatics10.1016/j.jbi.2016.03.02461:C(149-158)Online publication date: 1-Jun-2016
    • (2015)With best intentionsInformation Technology & People10.1108/ITP-12-2013-022328:2(246-280)Online publication date: 1-Jun-2015
    • (2015)Empirical Studies on Usability of mHealth AppsJournal of Medical Systems10.1007/s10916-014-0182-239:2(1-19)Online publication date: 1-Feb-2015
    • (2013)Don't talk to strangers!CHI '13 Extended Abstracts on Human Factors in Computing Systems10.1145/2468356.2468761(2337-2340)Online publication date: 27-Apr-2013
    • (2012)Evaluating game preference using the fun toolkit across culturesProceedings of the 26th Annual BCS Interaction Specialist Group Conference on People and Computers10.5555/2377916.2377971(386-391)Online publication date: 10-Sep-2012
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media