extended-abstract

Paper ladder: a rating scale to collect children's opinion in user studies

Authors:

Cristina Maria Sylla,

Ahmed Sabbir Arif,

Elena Márquez Segura,

Eva Irene BrooksAuthors Info & Claims

MobileHCI '17: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services

Article No.: 96, Pages 1 - 8

https://doi.org/10.1145/3098279.3122139

Published: 04 September 2017 Publication History

Abstract

We present Paper Ladder, a low-cost, light-weight, and paper-based variant of the Sticky Ladder rating scale to collect children's product preferences in user studies. This paper version revamps the original, effective but arguably impractical evaluation method by making it more accessible to researchers. In this paper, we show its potential with preliminary results from a pilot study, where 45 grade-2 children used a Paper Ladder to rate their preference for different types of correctness feedback in a math app. Results suggest that Paper Ladder could be an effective and valuable instrument for conducting user studies with young children.

References

[1]

Sharon Airey, Lydia Plowman, Daniel Connolly, and Rosemary Luckin. 2002. Rating children's enjoyment of toys, games and media. In Proceeding of the 3rd World Congress of the International Toy Research Association on Toys, Games and Media, 1--7.

[2]

Ahmed Sabbir Arif and Cristina Sylla. 2013. A comparative evaluation of touch and pen gestures for adult and child users. In Proceedings of the 12th International Conference on Interaction Design and Children - IDC '13, 392--395.

Digital Library

[3]

Ida Bark, Asbjørn Følstad, and Jan Gulliksen. 2006. Use and usefulness of HCI methods: results from an exploratory study among nordic HCI practitioners. In People and Computers XIX --- The Bigger Picture. Springer London, London, 201--217.

[4]

Alice Bell. 2007. Designing and testing questionnaires for children. Journal of Research in Nursing 12, 5: 461--469.

[5]

Christine T. Chambers. 2002. Developmental differences in children's use of rating scales. Journal of Pediatric Psychology 27, 1: 27--36.

[6]

Allison Druin. 1999. Cooperative inquiry: developing new technologies for children with children. In Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99, 592--599.

Digital Library

[7]

Allison Druin. 2002. The role of children in the design of new technology. Behaviour & Information Technology 21, 1: 1--25.

[8]

Lynne Hall, Colette Hume, and Sarah Tazzyman. 2016. Five degrees of happiness: effective smiley face Likert Scales for evaluating with children. In Proceedings of the The 15th International Conference on Interaction Design and Children - IDC '16, 311--321.

Digital Library

[9]

Libby Hanna, Kirsten Risden, and Kirsten Alexander. 1997. Guidelines for usability testing with children. interactions 4, 5: 9--14.

Digital Library

[10]

Malcolm Hill. 2006. Children's voices on ways of having a voice: children's and young people's perspectives on methods used in research and consultation. Childhood 13, 1: 69--89.

[11]

Jean Hunleth. 2011. Beyond on or with: questioning power dynamics and knowledge production in "child-oriented" research methodology. Childhood 18, 1: 81--93.

[12]

David Kirsh. 1995. Complementary strategies: why we use our hands when we think. In Proceedings of the 17th Annual Conference of the Cognitive Science Society - CogSci '95, 212--217.

[13]

Scott R. Klemmer, Björn Hartmann, and Leila Takayama. 2006. How bodies matter: five themes for interaction design. In Proceedings of the 6th ACM conference on Designing Interactive systems - DIS '06, 140.

Digital Library

[14]

H van Laerhoven, H J van der Zaag-Loonen, and B H F Derkx. 2004. A comparison of Likert scale and visual analogue scales as response options in children's questionnaires. Acta paediatrica 93, 6: 830--835.

[15]

David Mellor and Kathleen A. Moore. 2014. The use of Likert scales with children. Journal of Pediatric Psychology 39, 3: 369--379.

[16]

Janet C. Read and Stuart MacFarlane. 2002. Endurability, engagement and expectations: measuring children's fun. Interaction Design and Children 2: 1--23.

[17]

Janet C Read, Stuart MacFarlane, and Chris Casey. 2003. What's going on?: discovering what children understand about handwriting recognition interfaces. In Proceeding of the 2003 conference on Interaction design and children - IDC '03, 135.

Digital Library

[18]

Christine M Rubie-Davies and John A C Hattie. 2012. The dangers of extreme positive responses in Likert scales administered to young children. The International Journal of Educational and Psychological Assessment 11: 75--89.

[19]

Annika Wolff, Paul Mulholland, Zdenek Zdrahal, and Richard Joiner. 2007. Re-using digital narrative content in interactive games. International Journal of Human-Computer Studies 65, 3: 244--272.

Digital Library

[20]

Bieke Zaman, Vero Vanden Abeele,and Dirk De Grooff. 2013. Measuring product liking in preschool children: an evaluation of the Smileyometer and This or That methods. International Journal of Child-Computer Interaction 1, 2: 61--70.

Cited By

Menegazzi DSylla CPadovani S(2020)O design de um método para avaliação da experiência de interação em leitura mediada com livros infantis em dispositivos móveisBlucher Design Proceedings10.5151/9cidi-congic-4.0270(2916-2932)Online publication date: Jul-2020
https://doi.org/10.5151/9cidi-congic-4.0270
Menegazzi DSylla C(2020)Touch to Read: Investigating the Readers’ Interaction Experience in Mediated Reading to Design Story AppsInteractivity, Game Creation, Design, Learning, and Innovation10.1007/978-3-030-53294-9_43(588-600)Online publication date: 28-Jul-2020
https://doi.org/10.1007/978-3-030-53294-9_43
Sylla CMárquez Segura EDeWitt AArif ABrooks EArnedo JNacke LVanden Abeele VToups Dugas P(2019)Fiddling, Pointing, Hovering, and SlidingProceedings of the Annual Symposium on Computer-Human Interaction in Play10.1145/3311350.3347170(59-72)Online publication date: 17-Oct-2019
https://dl.acm.org/doi/10.1145/3311350.3347170
Show More Cited By

Index Terms

Paper ladder: a rating scale to collect children's opinion in user studies
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

Fiddling, Pointing, Hovering, and Sliding: Embodied Actions with Three Evaluation Tools for Children
CHI PLAY '19: Proceedings of the Annual Symposium on Computer-Human Interaction in Play

In user studies with children, it is important to use age appropriate evaluation tools to better understand their preferences, opinions, and thoughts. Here, we studied two accepted evaluation tools: The Five Degrees of Happiness, and the Sticky Ladder ...
Using the fun toolkit and other survey methods to gather opinions in child computer interaction
IDC '06: Proceedings of the 2006 conference on Interaction design and children

The paper begins with a review of some of the current literature on the use of survey methods with children. It then presents four known concerns with using survey methods for opinion gathering and reflects on how these concerns may impact on studies in ...
User eXperience Laddering with preschoolers: unveiling attributes and benefits of cuddly toy interfaces

In this paper, we suggest Laddering as a promising empirical method to evaluate the impact of tangibility on young children's user experiences. In the first part of this paper, we explain what Laddering is. We explicate the conceptual foundations of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MobileHCI '17: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services

September 2017

874 pages

ISBN:9781450350754

DOI:10.1145/3098279

General Chairs:
Matt Jones
Swansea University
,
Manfred Tscheligi
University of Salzburg and AIT
,
Program Chairs:
Yvonne Rogers
University College London
,
Roderick Murray-Smith
University of Glasgow

Copyright © 2017 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 September 2017

Check for updates

Author Tags

Qualifiers

Extended-abstract

Funding Sources

Conference

MobileHCI '17

Sponsor:

SIGCHI

MobileHCI '17: 19th International Conference on Human-Computer Interaction with Mobile Devices and Services

September 4 - 7, 2017

Vienna, Austria

Acceptance Rates

MobileHCI '17 Paper Acceptance Rate 45 of 224 submissions, 20%;

Overall Acceptance Rate 202 of 906 submissions, 22%

Upcoming Conference

MobileHCI '25

Sponsor:
sigchi

27th International Conference on Mobile Human-Computer Interaction

September 22 - 26, 2025

Sharm El-Sheikh , Egypt

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
166
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Menegazzi DSylla CPadovani S(2020)O design de um método para avaliação da experiência de interação em leitura mediada com livros infantis em dispositivos móveisBlucher Design Proceedings10.5151/9cidi-congic-4.0270(2916-2932)Online publication date: Jul-2020
https://doi.org/10.5151/9cidi-congic-4.0270
Menegazzi DSylla C(2020)Touch to Read: Investigating the Readers’ Interaction Experience in Mediated Reading to Design Story AppsInteractivity, Game Creation, Design, Learning, and Innovation10.1007/978-3-030-53294-9_43(588-600)Online publication date: 28-Jul-2020
https://doi.org/10.1007/978-3-030-53294-9_43
Sylla CMárquez Segura EDeWitt AArif ABrooks EArnedo JNacke LVanden Abeele VToups Dugas P(2019)Fiddling, Pointing, Hovering, and SlidingProceedings of the Annual Symposium on Computer-Human Interaction in Play10.1145/3311350.3347170(59-72)Online publication date: 17-Oct-2019
https://dl.acm.org/doi/10.1145/3311350.3347170
Rodríguez IKaryda MLucero AHerskovic V(2019)Aestimo: A Tangible Kit to Evaluate Older Adults’ User ExperienceHuman-Computer Interaction – INTERACT 201910.1007/978-3-030-29381-9_2(13-32)Online publication date: 23-Aug-2019
https://doi.org/10.1007/978-3-030-29381-9_2

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten