Kristina Dawidowsky
ABSTRACT
Assessing children’s literacy skills is a key requirement for successful learning. However, standardized assessments are almost exclusively available as paper-and-pencil tests, discarding digital testing’s advantages. In this article, we develop and evaluate two different tablet versions for a paper-based reading fluency test for German primary school children: one true-to-original design (TDId) and a tablet-optimized design (TDOpt). We investigate the reliability, equivalence, and user experience of the tablet-based versions by comparing it to each other and to the paper-based version in a user test with 21 fourth graders. The results suggest very high reliability of both tablet-based versions (r’s >.94). Children scored significantly higher on TDId than on the other tests while reading scores assessed with TDOpt matched the conventional test. Children preferred both tablet versions over the paper-based test, whereby TDOpt was consistently rated best. This study suggests that a tablet-optimized reading fluency test not only retains test reliability but is also more appealing to primary school children.
- Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2015. Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software 67, 1 (2015), 1–48. https://doi.org/10.18637/jss.v067.i01Google Scholar
Cross Ref
- Gal Ben-Yehudah and Yoram Eshet-Alkalai. 2021. Print versus digital reading comprehension tests: does the congruency of study and test medium matter?British Journal of Educational Technology 52, 1 (2021), 426–440. https://doi.org/10.1111/bjet.13014Google Scholar
- Jennifer Borse, Erica Robles, and Nancy Schwartz. 2006. Designing for Kids in the Digital Age: Summary of research and recommendations for designers. In The 1st interaction Design & Children Conference.Google Scholar
- Virginia Clinton. 2019. Reading from paper compared to screens: A systematic review and meta-analysis. Journal of Research in Reading 42, 2 (2019), 288–325. https://doi.org/10.1111/1467-9817.12269Google ScholarCross Ref
- William Jay Conover, Armando Jesús Guerrero-Serrano, and Víctor Gustavo Tercero-Gómez. 2018. An update on ‘a comparative study of tests for homogeneity of variance’. Journal of Statistical Computation and Simulation 88, 8(2018), 1454–1469. https://doi.org/10.1080/00949655.2018.1438437Google ScholarCross Ref
- Danielle Dahan Golan, Mirit Barzillai, and Tami Katzir. 2018. The effect of presentation mode on children’s reading preferences, performance, and self-evaluations. Computers & Education 126 (2018), 346–358. https://doi.org/10.1016/j.compedu.2018.08.001Google ScholarCross Ref
- Daniel Danner, Julian Aichholzer, and Beatrice Rammstedt. 2015. Acquiescence in personality questionnaires: Relevance, domain specificity, and stability. Journal of Research in Personality 57 (aug 2015), 119–130. https://doi.org/10.1016/j.jrp.2015.05.004Google Scholar
- Pablo Delgado, Cristina Vargas, Rakefet Ackerman, and Ladislao Salmerón. 2018. Don’t throw away your printed books: A meta-analysis on the effects of reading media on reading comprehension. Educational Research Review 25 (2018), 23–38. https://doi.org/10.1016/j.edurev.2018.09.003Google ScholarCross Ref
- Gary W. Evans, Gernot Brauchle, Aliya Haq, Rachel Stecker, Kimberly Wong, and Elan Shapiro. 2007. Young Children’s Environmental Attitudes and Behaviors. Environment and Behavior 39, 5 (sep 2007), 635–658. https://doi.org/10.1177/0013916506294252Google ScholarCross Ref
- Christian Fries. 2010. Grundlagen der Mediengestaltung - Konzeption, Ideenfindung, Bildaufbau, Farbe, Typografie, Interface Design(4 ed.). Carl Hanser Verlag GmbH Co KG, M. 175 pages.Google Scholar
- Katharina Galuschka and Gerd Schulte-Körne. 2016. The diagnosis and treatment of reading and/or spelling disorders in children and adolescents. Dtsch Arztebl Int. 113, 16 (2016), 279–286. https://doi.org/10.3238/arztebl.2016.0279Google Scholar
- D. Randy Garrison and Heather Kanuka. 2004. Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education 7, 2 (2004), 95–105. https://doi.org/10.1016/j.iheduc.2004.02.001Google ScholarCross Ref
- Amanda P. Goodwin, Sun Joo Cho, Dan Reynolds, Katherine Brady, and Jorge Salas. 2020. Digital Versus Paper Reading Processes and Links to Comprehension for Middle School Students. American Educational Research Journal 57, 4 (2020), 1837–1867. https://doi.org/10.3102/0002831219890300Google ScholarCross Ref
- Charles R Graham, Wendy Woodfield, and J Buckley Harrison. 2013. A framework for institutional adoption and implementation of blended learning in higher education. The Internet and Higher Education 18 (2013), 4–14. https://doi.org/10.1016/j.iheduc.2012.09.003Google ScholarCross Ref
- Geoffrey Haddock, Colin Foad, Victoria Saul, Will Brown, and Rose Thompson. 2020. The medium can influence the message: Print-based versus digital reading influences how people process different types of written information. British Journal of Psychology 111, 3 (2020), 443–459. https://doi.org/10.1111/bjop.12415Google ScholarCross Ref
- Vered Halamish and Elisya Elbaz. 2020. Children’s reading comprehension and metacomprehension on screen versus on paper. Computers & Education 145 (2020), 103737. https://doi.org/10.1016/j.compedu.2019.103737Google ScholarDigital Library
- Lynne Hall, Colette Hume, and Sarah Tazzyman. 2016. Five Degrees of Happiness: Effective Smiley Face Likert Scales for Evaluating with Children. In Proceedings of the The 15th International Conference on Interaction Design and Children - IDC ’16. Association for Computing Machinery, New York, NY, USA, 311–321. https://doi.org/10.1145/2930674.2930719Google ScholarDigital Library
- Zainab Hamza and Guarionex Salivia. 2015. Study of Touch Gestures Performance in Touch Devices by Young Children. International Journal on Recent and Innovation Trends in Computing and Communication 3, 3 (2015), 1395–1400. https://doi.org/10.17762/ijritcc.v3i3.4040Google ScholarCross Ref
- Heiko Holz, Katharina Brandelik, Benedikt Beuttler, Jochen Brandelik, and Manuel Ninaus. 2018. How to train your syllable stress awareness – A digital game-based intervention for German dyslexic children. International Journal of Serious Games 5, 3 (2018), 37–59. https://doi.org/10.17083/ijsg.v5i3.242Google ScholarCross Ref
- Ashley E Howe, Karen M Arnell, Raymond M Klein, Marc F Joanisse, and Rosemary Tannock. 2006. The ABCs of computerized naming: equivalency, reliability, and predictive validity of a computerized rapid automatized naming (RAN) task.Journal of neuroscience methods 151, 1 (feb 2006), 30–37. https://doi.org/10.1016/j.jneumeth.2005.07.014Google Scholar
- Yiren Kong, Young Sik Seo, and Ling Zhai. 2018. Comparison of reading performance on screen and on paper: A meta-analysis. Computers & Education 123 (2018), 138–149. https://doi.org/10.1016/j.compedu.2018.05.005Google ScholarDigital Library
- James A Kulik and J D Fletcher. 2016. Effectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review. Review of Educational Research 86, 1 (mar 2016), 42–78. https://doi.org/10.3102/0034654315581420Google ScholarCross Ref
- Alexandra Kuznetsova, Per B. Brockhoff, and Rune H. B. Christensen. 2017. lmerTest Package: Tests in Linear Mixed Effects Models. Journal of Statistical Software 82, 13 (2017). https://doi.org/10.18637/jss.v082.i13Google ScholarCross Ref
- Wolfgang Lenhard and Wolfgang Schneider. 2009. Diagnostik und Förderung des Leseverständnisses. Hogrefe Verlag, Göttingen.Google Scholar
- Wolfgang Lenhard, Wolfgang Schneider, Alexandra Lenhard, and Wolfgang Schneider. 2018. ELFE II: ein Leseverständnistest für Erst-bis Siebtklässler-Version II. Hogrefe. https://www.testzentrale.de/shop/ein-leseverstaendnistest-fuer-erst-bis-siebtklaessler.htmlGoogle Scholar
- Wolfgang Lenhard, Ulrich Schroeders, and Alexandra Lenhard. 2017. Equivalence of Screen Versus Print Reading Comprehension Depends on Task Complexity and Proficiency. Discourse Processes 54, 5-6 (2017), 427–445. https://doi.org/10.1080/0163853X.2017.1319653Google ScholarCross Ref
- Janine Liebal and Markus Exner. 2011. Usability für Kids - Ein Handbuch zur ergonomischen Gestaltung von Software und Websites für Kinder. Springer-Verlag, Berlin Heidelberg New York. https://doi.org/10.1007/978-3-8348-8206-6Google Scholar
- Jana Lüdtke, Eva Froehlich, Arthur M Jacobs, and Florian Hutzler. 2019. The SLS-Berlin: Validation of a German Computer-Based Screening Test to Measure Reading Proficiency in Early and Late Adulthood. Frontiers in Psychology 10, August (aug 2019), 1–23. https://doi.org/10.3389/fpsyg.2019.01682Google ScholarCross Ref
- Hans Mayringer and Heinz Wimmer. 2014. Salzburger Lese-Screening für die Schulstufen 2-9.Hogrefe.Google Scholar
- Miriam McBreen and Robert Savage. 2020. The Impact of Motivational Reading Instruction on the Reading Achievement and Motivation of Students: a Systematic Review and Meta-Analysis. Educational Psychology Review(2020). https://doi.org/10.1007/s10648-020-09584-4Google Scholar
- Jan L. Plass, Steffi Heidig, Elizabeth O. Hayward, Bruce D. Homer, and Enjoon Um. 2014. Emotional design in multimedia learning: Effects of shape and color on affect and learning. Learning and Instruction 29 (2014), 128 – 140. https://doi.org/10.1016/j.learninstruc.2013.02.006Google ScholarCross Ref
- Jan L Plass, Bruce D Homer, Andrew MacNamara, Teresa Ober, Maya C Rose, Shashank Pawar, Chris M Hovey, and Alvaro Olsen. 2020. Emotional design for digital games for learning: The effect of expression, color, shape, and dimensionality on the affective quality of game characters. Learning and Instruction 70 (2020), 101194. https://doi.org/10.1016/j.learninstruc.2019.01.005Google ScholarCross Ref
- Alexandre Porion, Xavier Aparicio, Olga Megalakaki, Alisson Robert, and Thierry Baccino. 2016. The impact of paper-based versus computerized presentation on text comprehension and memorization. Computers in Human Behavior 54 (2016), 569–576. https://doi.org/10.1016/j.chb.2015.08.002Google ScholarDigital Library
- Susanne Prediger and Lena Wessel. 2018. Brauchen mehrsprachige Jugendliche eine andere fach- und sprachintegrierte Förderung als einsprachige?Zeitschrift für Erziehungswissenschaft 21, 2(2018), 361–382. https://doi.org/10.1007/s11618-017-0785-8Google Scholar
- R Core Team. 2014. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.r-project.org/Google Scholar
- Janet C. Read. 2008. Validating the Fun Toolkit: an instrument for measuring children’s opinions of technology. Cognition, Technology & Work 10, 2 (apr 2008), 119–128. https://doi.org/10.1007/s10111-007-0069-9Google ScholarDigital Library
- Janet C. Read and Stuart MacFarlane. 2006. Using the fun toolkit and other survey methods to gather opinions in child computer interaction. Proceeding of the 2006 conference on Interaction design and children - IDC ’06 (2006), 81. https://doi.org/10.1145/1139073.1139096Google Scholar
- Janet C. Read, Stuart Macfarlane, and Chris Casey. 2002. Endurability, engagement and expectations: Measuring children’s fun. In Interaction design and children, Vol. 2. Shaker Publishing Eindhoven, 1–23. http://chici.uclan.ac.uk/references/endurability_engagement.pdfGoogle Scholar
- Luz Rello, Susana Bautista, Ricardo Baeza-Yates, Pablo Gervás, Raquel Hervás, and Horacio Saggion. 2013. One Half or 50%? An Eye-Tracking Study of Number Representation Readability. Springer Berlin Heidelberg, Berlin, Heidelberg, 229–245. https://doi.org/10.1007/978-3-642-40498-6_17Google Scholar
- Tobias Richter, Maj-Britt Isberner, Johannes Naumann, and Yvonne Kutzner. 2012. Prozessbezogene Diagnostik von Lesefähigkeiten bei Grundschulkindern. Zeitschrift für Pädagogische Psychologie 26, 4(2012), 313–331. https://doi.org/10.1024/1010-0652/a000079Google ScholarCross Ref
- Susanne Seifert and Lisa Paleczek. 2020. Development of a German Digital Assessment of Reading Comprehension in Grades 3-4. In European Conference on e-Learning. Academic Conferences International Limited. https://doi.org/10.34190/EEL.20.014Google Scholar
- Davino Silva Junior, Luciana Cidrim, Antonio Roazzi, and Francisco Madeiro. 2019. Digital version of the Rapid Automatized Naming (RAN): a contribution to early detection of reading problems in children. Revista CEFAC 21, 1 (2019). https://doi.org/10.1590/1982-0216/20192112518Google Scholar
- Lauren M Singer and Patricia A Alexander. 2017. Reading on Paper and Digitally: What the Past Decades of Empirical Research Reveal. Review of Educational Research 87, 6 (jul 2017), 1007–1041. https://doi.org/10.3102/0034654317722961Google ScholarCross Ref
- Nikita Soni, Aishat Aloba, Kristen S. Morga, Pamela J. Wisniewski, and Lisa Anthony. 2019. A Framework of Touchscreen Interaction Design Recommendations for Children (TIDRC): Characterizing the Gap between Research Evidence and Design Practice. In Proceedings of the 18th ACM International Conference on Interaction Design and Children(Boise, ID, USA) (IDC ’19). Association for Computing Machinery, New York, NY, USA, 419–431. https://doi.org/10.1145/3311927.3323149Google ScholarDigital Library
- Hildegunn Støle, Anne Mangen, and Knut Schwippert. 2020. Assessing children’s reading comprehension on paper and screen: A mode-effect study. Computers & Education 151 (2020), 103861. https://doi.org/10.1016/j.compedu.2020.103861Google ScholarCross Ref
Recommendations
Eye movements during reading and reading assessment in swedish school children: a new window on reading difficulties
ETRA '19: Proceedings of the 11th ACM Symposium on Eye Tracking Research & ApplicationsResearch during the last decades has demonstrated that eye tracking methodology is an advantageous tool to study reading. A substantial amount of eye movement research has resulted in improved understanding of the reading process in skilled adult ...
Comments