Lorans Alabood
Travis Dow
ABSTRACT
About 30% of autistic people are nonspeakers; they cannot use speech to communicate effectively. Pointing to letters on a letterboard held by a Communication and Regulation Partner (CRP) is one alternative method of expressive communication that some members of this population use. In the training of this method, a CRP delivers engaging and customized lessons. Additionally, the CRP provides regulatory, sensory, and attentional support and also works to strengthen the subject’s pointing skills. The goal of this training is to equip individuals with the required skills to be able to type independently. Recent studies have proposed using AR to provide opportunities for nonspeakers to practice the motor skills involved in typing. To use such systems, however, there needs to be a transition phase where a CRP teaches their subject how to interact with a virtual letterboard. In this paper, we explore the feasibility of using cross-reality, in which a CRP and nonspeaker can interact with the same virtual objects simultaneously, as a possible means of fostering this transition. We report a study involving 5 nonspeaking autistic subjects with diverse motor skills interacting using a virtual HoloLens 2 letterboard system we developed called HoloBoard. All subjects succeeded in pointing to letters correctly or spelling on the virtual board. We report process and design recommendations based on feedback obtained from subjects and their CRPs.
Supplemental Material
- Lorans Alabood, Zahra Aminolroaya, Dianna Yim, Omar Addam, and Frank Maurer. 2022. A systematic literature review of the Design Critique method. Information and Software Technology (2022), 107081. https://doi.org/10.1016/j.infsof.2022.107081Google Scholar
Digital Library
- Lorans Alabood, Evan Krul, Ali Shahidi, Vikram K Jaswal, Diwakar Krishnamurthy, and Mea Wang. 2022. HoloType-CR: Cross Reality Communication Training for Minimally Verbal Autistic Persons. In 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 187–190.Google Scholar
- Lorans Alabood and Frank Maurer. 2022. An iot-based immersive smart home system for seniors with neurocognitive disorders. In Proceedings of the 3d In-ternational Workshop on Empowering People in Dealing with Internet of Things Ecosystems, co-located with AVI, Rome, Italy, Vol. 3172. 10–15.Google Scholar
- Carmen Berenguer, Inmaculada Baixauli, Soledad Gómez, María de El Puig Andrés, and Simona De Stasio. 2020. Exploring the Impact of Augmented Reality in Children and Adolescents with Autism Spectrum Disorder: A Systematic Review. International Journal of Environmental Research and Public Health 17, 17, Article 6143 (2020).Google Scholar
- David R Beukelman, Pat Mirenda, 1998. Augmentative and alternative communication. Paul H. Brookes Baltimore.Google Scholar
- Eli Blevis, Youn-kyung Lim, David Roedl, and Erik Stolterman. 2007. Using design critique as research to link sustainability and interactive technologies. In International Conference on Online Communities and Social Computing. Springer, 22–31.Google ScholarDigital Library
- Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.Google Scholar
- Chad Cardin. 2021. A nonspeaking valedictorian with autism gives her college’s commencement speech. Canadian Broadcasting Corporation News (2021).Google Scholar
- Bill Chappell. 2022. A nonspeaking valedictorian with autism gives her college’s commencement speech. National Public Radio NPR (2022).Google Scholar
- Charlotte DiStefano, Wendy Shih, Ann Kaiser, Rebecca Landa, and Connie Kasari. 2016. Communication growth in minimally verbal children with ASD: The importance of interaction. Autism Research 9, 10 (2016), 1093–1102.Google ScholarCross Ref
- Travis Dow, Lorans Alabood, and Diwakar Krishnamurthy. 2023. SimpleShare unity package. https://github.com/Krishnamurthy-AR-Project/SimpleShare.Google Scholar
- Christopher Frauenberger, Judith Good, Alyssa Alcorn, and Helen Pain. 2013. Conversing through and about technologies: Design critique as an opportunity to engage children with autism and broaden research (er) perspectives. International Journal of Child-Computer Interaction 1, 2 (2013), 38–49.Google ScholarCross Ref
- Jennifer B Ganz, Theresa L Earles-Vollrath, Amy K Heath, Richard I Parker, Mandy J Rispoli, and Jaime B Duran. 2012. A meta-analysis of single case research studies on aided augmentative and alternative communication systems with individuals with autism spectrum disorders. Journal of autism and developmental disorders 42, 1 (2012), 60–74.Google ScholarCross Ref
- Naoki Higashida. 2013. The reason I jump: The inner voice of a thirteen-year-old boy with autism. Knopf Canada.Google Scholar
- Vikram K Jaswal, Allison Wayne, and Hudson Golino. 2020. Eye-tracking reveals agency in assisted autistic communication. Scientific reports 10, 1 (2020), 1–10.Google Scholar
- Kamran Khowaja, Bilikis Banire, Dena Al-Thani, Mohammed Tahri Sqalli, Aboubakr Aqle, Asadullah Shah, and Siti Salwah Salim. 2020. Augmented Reality for Learning of Children and Adolescents With Autism Spectrum Disorder (ASD): A Systematic Review. IEEE Access 8 (2020), 78779–78807.Google ScholarCross Ref
- Diwakar Krishnamurthy, Vikram Jaswal, Ahmadreza Nazari, Ali Shahidi, Pranav Subbaraman, and Mea Wang. 2022. HoloType: Lived Experience Based Communication Training for Nonspeaking Autistic People. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–6.Google Scholar
- Elizabeth R Lorah, Christine Holyfield, Jessica Miller, Brenna Griffen, and Cody Lindbloom. 2022. A systematic review of research comparing mobile technology speech-generating devices to other AAC modes with individuals with autism spectrum disorder. Journal of Developmental and Physical Disabilities 34, 2 (2022), 187–210.Google ScholarCross Ref
- Edlyn Vallejo Peña. 2019. Communication Alternatives in Autism: Perspectives on Typing and Spelling Approaches for the Nonspeaking. McFarland.Google Scholar
- Edlyn Vallejo Peña. 2019. Leaders Around Me: Autobiographies of Autistics who type, point, & spell to communicate.Google Scholar
- David James Savarese. 2019. Coming to my senses. Autism in Adulthood 1, 2 (2019), 90–92.Google ScholarCross Ref
- Monica Tentori, Lizbeth Escobedo, and Gabriela Balderas. 2015. A Smart Environment for Children with Autism. IEEE Pervasive Computing 14, 2 (2015), 42–50.Google ScholarDigital Library
- Janna Van Grunsven and Sabine Roeser. 2022. AAC technology, autism, and the empathic turn. Social Epistemology 36, 1 (2022), 95–110.Google ScholarCross Ref
Index Terms
- Can Cross-Reality Help Nonspeaking Autistic People Transition to AR Typing?
Recommendations
Interactive AR Applications for Nonspeaking Autistic People? - A Usability Study
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing SystemsAbout one-third of autistic people are nonspeaking, and most are never provided access to an effective alternative to speech. Thoughtfully designed AR applications could provide members of this population with structured learning opportunities, ...
Through Tinted Eyeglasses
EIC Roy Want introduces the special issue on cross-reality environments and discusses alternate realities including virtual reality, augmented reality, embodied virtuality, cross-reality, and mixed reality.
Personalizing an AR-based Communication System for Nonspeaking Autistic Users
IUI '24: Proceedings of the 29th International Conference on Intelligent User InterfacesNonspeaking autistic individuals ("nonspeakers") represent about one-third of the autistic population, and most are never provided with an effective alternative to speech, hindering their educational, employment, and social opportunities. Some ...
Comments