Danlu Fei
Ze Gao
ABSTRACT
Children with dyslexia face extra challenges in reading and writing words. They need more learning exercises than children with typical development to acquire vocabulary, which is often repetitive and daunting. Research has shown that combining visuospatial information in practices helped children with dyslexia memorize words, especially the real-world physical context. Nevertheless, the existing word recognition and spelling training games for children with dyslexia were not able to leverage children’s immediate vicinity. Therefore, we designed an augmented reality mobile game, CollectiAR, that uses computer vision to identify objects in the player’s immediate vicinity and direct the player to learn words for these objects. Our formative study with two elementary school teachers and a first-grade pupil found that CollectiAR has the potential to be an integral part of teachers’ instructional design and an engaging way for pupils to practice vocabulary exercises. Our teacher participants suggested that CollectiAR provide interfaces for teachers to participate in the game content design and computer vision model correction.
- Mary Alt, Tiffany Hogan, Samuel Green, Shelley Gray, Kathryn Cabbage, and Nelson Cowan. 2017. Word learning deficits in children with dyslexia. Journal of Speech, Language, and Hearing Research 60, 4 (April 2017), 1012–1028. https://doi.org/10.1044/2016_JSLHR-L-16-0036Google Scholar
Cross Ref
- Richard Bartle. 1996. Hearts, clubs, diamonds, spades: Players who suit MUDs. (1996).Google Scholar
- Sarah Brown. 2011. Dyslexia Games - Word Hunt 1. CreateSpace.Google Scholar
- Samuel Budd, Emma C Robinson, and Bernhard Kainz. 2021. A survey on active learning and human-in-the-loop deep learning for medical image analysis. Medical Image Analysis 71 (2021), 102062.Google ScholarCross Ref
- Liz Burton. 2016. Helping your student with dyslexia: Learn 5 strategies to rely on. https://www.dyslexic.com/helping-your-student-with-dyslexia-learn-5-strategies-to-rely-on/Google Scholar
- Min Fan, Jianyu Fan, Alissa N Antle, Sheng Jin, Dongxu Yin, and Philippe Pasquier. 2019. Character alive: A tangible reading and writing system for Chinese children at-risk for dyslexia. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1–6.Google ScholarDigital Library
- Yuichiro Fujimoto, Goshiro Yamamoto, Takafumi Taketomi, Jun Miyazaki, and Hirokazu Kato. 2012. Relationship between features of augmented reality and user memorization. In 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE. https://doi.org/10.1109/ismar.2012.6402573Google ScholarDigital Library
- Tushar Gupta, Leila Aflatoony, and Lynette Leonard. 2021. Augmenta11y: A reading assistant application for children with dyslexia. In The 23rd International ACM SIGACCESS Conference on Computers and Accessibility. 1–3.Google ScholarDigital Library
- Marc Hassenzahl. 2008. User experience: (UX) towards an experiential perspective on product quality. In Proceedings of the 20th Conference on l’Interaction Homme-Machine. 11–15.Google ScholarDigital Library
- Ting-Chia Hsu. 2017. Learning English with augmented reality: Do learning styles matter?Computers & Education 106 (2017), 137–149. https://doi.org/10.1016/j.compedu.2016.12.007Google Scholar
- Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara, Alireza Fathi, Ian Fischer, Zbigniew Wojna, Yang Song, Sergio Guadarrama, and Kevin Murphy. 2017. Speed/accuracy trade-offs for modern convolutional object detectors. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Honolulu, HI, 3296–3297. https://doi.org/10.1109/CVPR.2017.351Google ScholarCross Ref
- Adam Ibrahim, Brandon Huynh, Jonathan Downey, Tobias Höllerer, Dorothy Chun, and John O’donovan. 2018. Arbis pictus: A study of vocabulary learning with augmented reality. IEEE transactions on visualization and computer graphics 24, 11(2018), 2867–2874.Google Scholar
- Monika Kast, Gian-Marco Baschera, Markus Gross, Lutz Jäncke, and Martin Meyer. 2011. Computer-based learning of spelling skills in children with and without dyslexia. Annals of Dyslexia 61, 2 (2011), 177–200. https://doi.org/10.1007/s11881-011-0052-2Google ScholarCross Ref
- James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114, 13(2017), 3521–3526.Google ScholarCross Ref
- Chen-Yu Lee and Simon Osindero. 2016. Recursive recurrent nets with attention modeling for ocr in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2231–2239.Google ScholarCross Ref
- Pranay Mathur, Aman Gill, Aayush Yadav, Anurag Mishra, and Nand Kumar Bansode. 2017. Camera2Caption: A real-time image caption generator. In 2017 international conference on computational intelligence in data science (ICCIDS). IEEE, 1–6.Google ScholarCross Ref
- Pascal Meier and Frank Teuteberg. 2021. Augmenting humans in the loop: Towards an augmented reality object labeling application for crowdsourcing Communities. Innovation Through Information Systems 2 (2021), 198.Google Scholar
- Niantic. 2016. Pokémon GO game. https://pokemongolive.com/en/Google Scholar
- Luz Rello, Clara Bayarri, and Azuki Gorriz. 2012. What is wrong with this word? Dyseggxia: A game for children with dyslexia. In Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility. ACM Press, Boulder, Colorado, USA, 219. https://doi.org/10.1145/2384916.2384962Google ScholarDigital Library
- Luz Rello, Arturo Macias, Mariía Herrera, Camila de Ros, Enrique Romero, and Jeffrey P. Bigham. 2017. DytectiveU: A game to train the difficulties and the strengths of children with dyslexia. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, Baltimore Maryland USA, 319–320. https://doi.org/10.1145/3132525.3134773Google ScholarDigital Library
- Glenda Revelle, Emily Reardon, Kristin Cook, Lori Takeuchi, Rafael Ballagas, Koichi Mori, Hiroshi Horii, Hayes Raffle, Maria Sandberg, and Mirjana Spasojevic. 2014. Electric agents: Combining collaborative mobile augmented reality and web-based video to reinvent interactive television. Computers in Entertainment 12, 3 (2014), 1–21. https://doi.org/10.1145/2702109.2633413Google ScholarDigital Library
- Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. MobileNetV2: Inverted residuals and linear bottlenecks. (2018). https://doi.org/10.48550/ARXIV.1801.04381Google Scholar
- Liliane Sprenger-Charolles, Linda S. Siegel, Juan E. Jiménez, and Johannes C. Ziegler. 2011. Prevalence and reliability of phonological, surface, and mixed profiles in dyslexia: A review of studies conducted in languages varying in orthographic depth. Scientific Studies of Reading 15, 6 (2011), 498–521. https://doi.org/10.1080/10888438.2010.524463Google ScholarCross Ref
- Feng Tian, Fei Lv, Jingtao Wang, Hongan Wang, Wencan Luo, Matthew Kam, Vidya Setlur, Guozhong Dai, and John Canny. 2010. Let’s play chinese characters: Mobile learning approaches via culturally inspired group games. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1603–1612.Google ScholarDigital Library
- Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3156–3164.Google ScholarCross Ref
- Zikai Alex Wen, Erica Silverstein, Yuhang Zhao, Anjelika Lynne Amog, Katherine Garnett, and Shiri Azenkot. 2020. Teacher views of math e-learning tools for students with specific learning disabilities. In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility. 1–13.Google ScholarDigital Library
- Debra Wise and Sandra Forrest. 2003. Great big book of children’s games: Over 450 indoor and outdoor games for kids. McGraw-Hill.Google Scholar
Index Terms
- CollectiAR: Computer Vision-Based Word Hunt for Children with Dyslexia
Recommendations
A computer-based method to improve the spelling of children with dyslexia
ASSETS '14: Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibilityIn this paper we present a method which aims to improve the spelling of children with dyslexia through playful and targeted exercises. In contrast to previous approaches, our method does not use correct words or positive examples to follow, but presents ...
ARLexic game: an augmented reality-based serious game for training of dyslexic and dysgraphic children
AbstractOver the years, researchers have discovered increased problems among children related to reading and writing. Dyslexia and Dysgraphia are the most common problems they try to solve with various paper-based activities and gaming interventions. But ...
Developing effective educative games for Arabic children primarily dyslexics
Since the early stages of schooling, many children are exposed to different learning disabilities, usually manifest as dyslexia, dysgraphia, and dyscalculia. Those disabilities impact on the normal academic achievement of the child and may even affect ...
Comments