skip to main content
research-article

Players and Performance: Opportunities for Social Interaction with Augmented Tabletop Games at Centres for Children with Autism

Published: 14 November 2022 Publication History

Abstract

This research aimed to investigate how children with autism interacted with rich audio and visual augmented reality (AR) tabletop games. Based on in-depth needs analysis facilitated through autism centers in China, we designed and developed MagicBLOCKS, a series of tabletop AR interactive games for children with autism. We conducted a four-week field study with 15 male children. We found that the interactive dynamics in games were rewarding and played critical roles in motivation and sustained interest. In addition, based on post-hoc interviews and video analysis with expert therapists, we found that MagicBLOCKS provided opportunities for children with autism to engage with each other through player performances and audience interactions with episodes of cooperation and territoriality. We discuss the limitations and the insights offered by this research.

Supplementary Material

Teaser (iss22main-id5568-p-teaser.mp4)
This is a teaser video of my talk at ISS 2022. This research aimed to investigate how children with autism interacted with rich audio and visual augmented reality (AR) tabletop games. Based on in-depth needs analysis facilitated through autism centers in China, we designed and developed MagicBLOCKS, a series of tabletop AR interactive games for children with autism. We conducted a four-week field study with 15 male children. We found that the interactive dynamics in games were rewarding and played critical roles in motivation and sustained interest. In addition, based on post-hoc interviews and video analysis with expert therapists, we found that MagicBLOCKS provided opportunities for children with autism to engage with each other through player performances and audience interactions with episodes of cooperation and territoriality. We discuss the limitations and the insights offered by this research.

References

[1]
Fouzia Khursheed Ahmad. 2015. Use of assistive technology in inclusive education: making room for diverse learning needs. Transcience 6, 2 ( 2015 ), 62-77.
[2]
A. Battocchi, F. Pianesi, D. Tomasini, M. Zancanaro, G. Esposito, P. Venuti, A. Ben Sasson, E. Gal, and P. L. Weiss. 2009. Collaborative Puzzle Game: A Tabletop Interactive Game for Fostering Collaboration in Children with Autism Spectrum Disorders (ASD). In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (Banf, Alberta, Canada) (ITS '09). Association for Computing Machinery, New York, NY, USA, 197-204. https: //doi.org/10.1145/1731903.1731940
[3]
Behavior Analyst Certification Board. 2014. Professional and ethical compliance code for behavior analysts.
[4]
Sven Bölte, Ofer Golan, Matthew S Goodwin, and Lonnie Zwaigenbaum. 2010. What can innovative technologies do for autism spectrum disorders?
[5]
Chien-Hsu Chen, I-Jui Lee, and Ling-Yi Lin. 2016. Augmented reality-based video-modeling storybook of nonverbal facial cues for children with autism spectrum disorder to improve their perceptions and judgments of facial expressions and emotions. Computers in Human Behavior 55 ( 2016 ), 477-485. https://doi.org/10.1016/j.chb. 2015. 09.033
[6]
Yan-Fang Chen. 2002. Chinese classification of mental disorders (CCMD-3): towards integration in international classification. Psychopathology 35, 2-3 ( 2002 ), 171-175. https://doi.org/10.1159/000065140
[7]
Jaime Craig and Simon Baron-Cohen. 1999. Creativity and imagination in autism and Asperger syndrome. Journal of autism and developmental disorders 29, 4 ( 1999 ), 319-326. https://doi.org/10.1023/A:1022163403479
[8]
Lizbeth Escobedo, Catalina Ibarra, Jehu Hernandez, Mariana Alvelais, and Monica Tentori. 2014. Smart Objects to Support the Discrimination Training of Children with Autism. Personal Ubiquitous Comput. 18, 6 (Aug. 2014 ), 1485-1497. https://doi.org/10.1007/s00779-013-0750-3
[9]
Lizbeth Escobedo, David H. Nguyen, LouAnne Boyd, Sen Hirano, Alejandro Rangel, Daniel Garcia-Rosas, Monica Tentori, and Gillian Hayes. 2012. MOSOCO: A Mobile Assistive Tool to Support Children with Autism Practicing Social Skills in Real-Life Situations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) ( CHI '12). Association for Computing Machinery, New York, NY, USA, 2589-2598. https: //doi.org/10.1145/2207676.2208649
[10]
Charles B Ferster. 1964. Positive reinforcement and behavioral deficits of autistic children. In Conditioning techniques in clinical practice and research. Springer, 255-274. https://doi.org/10.1007/978-3-662-39876-0_23
[11]
Ben Fletcher-Watson. 2015. Relaxed performance: audiences with autism in mainstream theatre. Scottish Journal of Performance 2, 2 ( 2015 ). https://doi.org/10.14439/sjop. 2015. 0202.04
[12]
Ben Fletcher-Watson and Shaun May. 2018. Enhancing relaxed performance: evaluating the Autism Arts Festival. Research in Drama Education: The Journal of Applied Theatre and Performance 23, 3 ( 2018 ), 406-420. https://doi.org/10. 1080/13569783. 2018.1468243
[13]
Christopher Frauenberger, Julia Makhaeva, and Katta Spiel. 2016. Designing Smart Objects with Autistic Children: Four Design Exposès. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) ( CHI '16). Association for Computing Machinery, New York, NY, USA, 130-139. https://doi.org/10. 1145/2858036.2858050
[14]
Christopher Frauenberger, Katta Spiel, and Julia Makhaeva. 2019. Thinking outsideTheBox-designing smart things with autistic children. International Journal of Human-Computer Interaction 35, 8 ( 2019 ), 666-678. https://doi.org/10. 1080/10447318. 2018.1550177
[15]
Christopher Gillberg. 2002. A guide to Asperger syndrome. Cambridge University Press.
[16]
Erving Gofman. 1959. The presentation of self in everyday life.
[17]
Carol Grbich. 2012. Qualitative data analysis: An introduction. Sage.
[18]
Antonia F de C Hamilton and Frida Lind. 2016. Audience efects: what can they tell us about social neuroscience, theory of mind and autism? Culture and Brain 4, 2 ( 2016 ), 159-177. https://doi.org/10.1007/s40167-016-0044-5
[19]
Sen H. Hirano, Michael T. Yeganyan, Gabriela Marcu, David H. Nguyen, Lou Anne Boyd, and Gillian R. Hayes. 2010. VSked: Evaluation of a System to Support Classroom Activities for Children with Autism. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) ( CHI '10). Association for Computing Machinery, New York, NY, USA, 1633-1642. https://doi.org/10.1145/1753326.1753569
[20]
Ann X Huang, Meixiang Jia, and John J Wheeler. 2013. Children with autism in the People's Republic of China: Diagnosis, legal issues, and educational services. Journal of Autism and Developmental Disorders 43, 9 ( 2013 ), 1991-2001. https://doi.org/10.1007/s10803-012-1722-6
[21]
Kamran Khowaja, Bilikis Banire, Dena Al-Thani, Mohammed Tahri Sqalli, Aboubakr Aqle, Asadullah Shah, and Siti Salwah Salim. 2020. Augmented Reality for Learning of Children and Adolescents With Autism Spectrum Disorder (ASD): A Systematic Review. IEEE Access 8 ( 2020 ), 78779-78807. https://doi.org/10.1109/ACCESS. 2020.2986608
[22]
Aaron Kline, Catalin Voss, Peter Washington, Nick Haber, Hessey Schwartz, Qandeel Tariq, Terry Winograd, Carl Feinstein, and Dennis P. Wall. 2019. Superpower Glass. GetMobile: Mobile Comp. and Comm. 23, 2 (Nov. 2019 ), 35-38. https://doi.org/10.1145/3372300.3372308
[23]
Daniel Klinkhammer, Magdalena Mateescu, Carmen Zahn, and Harald Reiterer. 2018. Mine, Yours, Ours: Coordination through Workspace Arrangements and Territoriality in Tabletop Interaction. In Proceedings of the 17th International Conference on Mobile and Ubiquitous Multimedia (Cairo, Egypt) (MUM 2018 ). Association for Computing Machinery, New York, NY, USA, 171-182. https://doi.org/10.1145/3282894.3282902
[24]
Hubert Knoblauch, Bernt Schnettler, Jürgen Raab, and Hans-Georg Soefner. 2012. Video analysis: methodology and methods. Peter Lang.
[25]
Hirokazu Kumazaki, Taro Muramatsu, Kazuki Kobayashi, Tetsuyou Watanabe, Kazunori Terada, Haruhiro Higashida, Teruko Yuhi, Masaru Mimura, and Mitsuru Kikuchi. 2020. Feasibility of autism-focused public speech training using a simple virtual audience for autism spectrum disorder. Psychiatry and clinical neurosciences 74, 2 ( 2020 ), 124-131. https://doi.org/10.1111/pcn.12949
[26]
Beibin Li, Adham Atyabi, Minah Kim, Erin Barney, Amy Yeojin Ahn, Yawen Luo, Madeline Aubertine, Sarah Corrigan, Tanya St. John, Quan Wang, Marilena Mademtzi, Mary Best, and Frederick Shic. 2018. Social Influences on Executive Functioning in Autism: Design of a Mobile Gaming Platform. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1-13. https://doi.org/10.1145/3173574.3174017
[27]
Daniele Mazzei, Nicole Lazzeri, Lucia Billeci, Roberta Igliozzi, Alice Mancini, Arti Ahluwalia, Filippo Muratori, and Danilo De Rossi. 2011. Development and evaluation of a social robot platform for therapy in autism. In 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 4515-4518. https://doi.org/10.1109/ IEMBS. 2011.6091119
[28]
Kenneth W Merrell and Gretchen Gimpel. 2014. Social skills of children and adolescents: Conceptualization, assessment, treatment. Psychology Press. https://doi.org/10.4324/9781315805757
[29]
Geir Overskeid. 2016. Power and autistic traits. Frontiers in Psychology 7 ( 2016 ), 1290.
[30]
Eleni Papadaki, Stavroula Ntoa, Ilia Adami, and Constantine Stephanidis. 2017. Let's cook: An augmented reality system towards developing cooking skills for children with cognitive impairments. In International conference on smart objects and technologies for social good. Springer, 237-247. https://doi.org/10.1007/978-3-319-76111-4_24
[31]
Linda Pring, Nicola Ryder, Laura Crane, and Beate Hermelin. 2012. Creativity in savant artists with autism. Autism 16, 1 ( 2012 ), 45-57. https://doi.org/10.1177/1362361311403783
[32]
Daniel J Ricks and Mark B Colton. 2010. Trends and considerations in robot-assisted autism therapy. In 2010 IEEE international conference on robotics and automation. IEEE, 4354-4359. https://doi.org/10.1109/ROBOT. 2010.5509327
[33]
Bernard Rimland. 1964. Infantile autism. ( 1964 ).
[34]
Mandy J Rispoli. 2013. Verbal Behavior Milestones Assessment and Placement Program VB-MAPP. Encyclopedia of Special Education: A Reference for the Education of Children, Adolescents, and Adults with Disabilities and Other Exceptional Individuals ( 2013 ). https://doi.org/10.1002/9781118660584.ese2479
[35]
Ben Robins, Kerstin Dautenhahn, Rene Te Boekhorst, and Aude Billard. 2005. Robotic assistants in therapy and education of children with autism: can a small humanoid robot help encourage social interaction skills? Universal access in the information society 4, 2 ( 2005 ), 105-120. https://doi.org/10.1007/s10209-005-0116-3
[36]
Stacey D. Scott, M. Sheelagh T. Carpendale, and Kori Inkpen. 2004. Territoriality in Collaborative Tabletop Workspaces. In Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work (Chicago, Illinois, USA) ( CSCW '04). Association for Computing Machinery, New York, NY, USA, 294-303. https://doi.org/10.1145/1031607.1031655
[37]
Jessica Singer-Dudek, Victoria Sterkin, and Lisa Gold Linden. 2021. Audience Control and the Emission of Stereotypy and Social Verbal Exchanges in Children With Autism and Developmental Disabilities. Behavior Analysis in Practice 14, 1 ( 2021 ), 75-85.
[38]
Karanya Sitdhisanguan, Nopporn Chotikakamthorn, Ajchara Dechaboon, and Patcharaporn Out. 2012. Using tangible user interfaces in computer-based training systems for low-functioning autistic children. Personal and Ubiquitous Computing 16, 2 ( 2012 ), 143-155. https://doi.org/10.1007/s00779-011-0382-4
[39]
Tobias Sonne and Mads Møller Jensen. 2016. ChillFish: A Respiration Game for Children with ADHD. In Proceedings of the TEI '16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (Eindhoven, Netherlands) (TEI '16). Association for Computing Machinery, New York, NY, USA, 271-278. https://doi.org/10.1145/2839462.2839480
[40]
Autism Speaks. 2011. What is autism. Retrieved on November 17 ( 2011 ), 2011.
[41]
Katta Spiel, Christopher Frauenberger, Os Keyes, and Geraldine Fitzpatrick. 2019. Agency of Autistic Children in Technology Research-A Critical Literature Review. ACM Trans. Comput.-Hum. Interact. 26, 6, Article 38 ( Nov. 2019 ), 40 pages. https://doi.org/10.1145/3344919
[42]
Cady M. Stanton, Peter H. Kahn Jr., Rachel L. Severson, Jolina H. Ruckert, and Brian T. Gill. 2008. Robotic Animals Might Aid in the Social Development of Children with Autism. In Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction (Amsterdam, The Netherlands) (HRI '08). Association for Computing Machinery, New York, NY, USA, 271-278. https://doi.org/10.1145/1349822.1349858
[43]
Issey Takahashi, Mika Oki, Baptiste Bourreau, Itaru Kitahara, and Kenji Suzuki. 2018. FUTUREGYM: A gymnasium with interactive floor projection for children with special needs. International Journal of Child-Computer Interaction 15 ( 2018 ), 37-47. https://doi.org/10.1016/j.ijcci. 2017. 12.002
[44]
James W Tanaka and Andrew Sung. 2016. The “eye avoidance” hypothesis of autism face processing. Journal of autism and developmental disorders 46, 5 ( 2016 ), 1538-1552. https://doi.org/10.1007/s10803-013-1976-7
[45]
Monica Tentori and Gillian R. Hayes. 2010. Designing for Interaction Immediacy to Enhance Social Skills of Children with Autism. In Proceedings of the 12th ACM International Conference on Ubiquitous Computing (Copenhagen, Denmark) ( UbiComp '10). Association for Computing Machinery, New York, NY, USA, 51-60. https://doi.org/10.1145/1864349. 1864359
[46]
Jennifer Thom-Santelli, Dan R. Cosley, and Geri Gay. 2009. What's Mine is Mine: Territoriality in Collaborative Authoring. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) ( CHI '09). Association for Computing Machinery, New York, NY, USA, 1481-1484. https://doi.org/10.1145/1518701.1518925
[47]
Norman Triplett. 1898. The dynamogenic factors in pacemaking and competition. The American journal of psychology 9, 4 ( 1898 ), 507-533. https://doi.org/10.2307/1412188
[48]
Jorien Van Hoorn, Eric Van Dijk, Eveline A Crone, Lex Stockmann, and Carolien Riefe. 2017. Peers influence prosocial behavior in adolescent males with autism spectrum disorders. Journal of Autism and Developmental Disorders 47, 7 ( 2017 ), 2225-2237. https://doi.org/10.1007/s10803-017-3143-z
[49]
Lilia Villafuerte, Milena Markova, and Sergi Jorda. 2012. Acquisition of Social Abilities through Musical Tangible User Interface: Children with Autism Spectrum Condition and the Reactable. In CHI '12 Extended Abstracts on Human Factors in Computing Systems (Austin, Texas, USA) ( CHI EA '12). Association for Computing Machinery, New York, NY, USA, 745-760. https://doi.org/10.1145/2212776.2212847
[50]
Catalin Voss, Peter Washington, Nick Haber, Aaron Kline, Jena Daniels, Azar Fazel, Titas De, Beth McCarthy, Carl Feinstein, Terry Winograd, and Dennis Wall. 2016. Superpower Glass: Delivering Unobtrusive Real-Time Social Cues in Wearable Systems. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct (Heidelberg, Germany) (UbiComp '16). Association for Computing Machinery, New York, NY, USA, 1218-1226. https://doi.org/10.1145/2968219.2968310
[51]
Peter Washington, Catalin Voss, Aaron Kline, Nick Haber, Jena Daniels, Azar Fazel, Titas De, Carl Feinstein, Terry Winograd, and Dennis Wall. 2017. SuperpowerGlass: A Wearable Aid for the At-Home Therapy of Children with Autism. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 112 ( Sept. 2017 ), 22 pages. https: //doi.org/10.1145/3130977
[52]
Qin Wu, Chenmei Yu, Yanjun Chen, Jiayu Yao, Xi Wu, Xiaolan Peng, and Teng Han. 2020. Squeeze the Ball: Designing an Interactive Playground towards Aiding Social Activities of Children with Low-Function Autism. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) ( CHI '20). Association for Computing Machinery, New York, NY, USA, 1-14. https://doi.org/10.1145/3313831.3376888
[53]
Xiaoqian Xie, Jiashuo Cao, Jiayu Yao, Lunyu Shang, Qianru Liu, Jinlun Lin, and Qin Wu. 2020. MaskMe: Using Masks to Design Collaborative Games for Helping Children with Autism Make Eye Contact. In Companion Proceedings of the 2020 Conference on Interactive Surfaces and Spaces (Virtual Event, Portugal) ( ISS '20). Association for Computing Machinery, New York, NY, USA, 29-32. https://doi.org/10.1145/3380867.3426206
[54]
Gulnoza Yakubova, Melissa A Defayette, Briella Baer Chen, and Anthony L Proulx. 2021. The Use of Augmented Reality Interventions to Provide Academic Instruction for Children with Autism, Intellectual, and Developmental Disabilities: an Evidence-Based Systematic Review. Review Journal of Autism and Developmental Disorders ( 2021 ), 1-17. https://doi.org/10.1007/s40489-021-00287-2

Cited By

View all
  • (2024)Fostering Motivation: Exploring the Impact of ICTs on the Learning of Students with AutismChildren10.3390/children1101011911:1(119)Online publication date: 18-Jan-2024
  • (2024)StarRescue: the Design and Evaluation of A Turn-Taking Collaborative Game for Facilitating Autistic Children's Social SkillsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642829(1-19)Online publication date: 11-May-2024
  • (2024)Understanding Neurodiverse Social Play Between Autistic and Non-Autistic ChildrenProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642809(1-16)Online publication date: 11-May-2024

Index Terms

  1. Players and Performance: Opportunities for Social Interaction with Augmented Tabletop Games at Centres for Children with Autism

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image Proceedings of the ACM on Human-Computer Interaction
      Proceedings of the ACM on Human-Computer Interaction  Volume 6, Issue ISS
      December 2022
      746 pages
      EISSN:2573-0142
      DOI:10.1145/3554337
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 14 November 2022
      Published in PACMHCI Volume 6, Issue ISS

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Accessibility
      2. Assistive Technology
      3. Audience Socializing
      4. Augmented Reality(AR)
      5. Children with autism
      6. Tabletop Interaction

      Qualifiers

      • Research-article

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)168
      • Downloads (Last 6 weeks)20
      Reflects downloads up to 20 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Fostering Motivation: Exploring the Impact of ICTs on the Learning of Students with AutismChildren10.3390/children1101011911:1(119)Online publication date: 18-Jan-2024
      • (2024)StarRescue: the Design and Evaluation of A Turn-Taking Collaborative Game for Facilitating Autistic Children's Social SkillsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642829(1-19)Online publication date: 11-May-2024
      • (2024)Understanding Neurodiverse Social Play Between Autistic and Non-Autistic ChildrenProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642809(1-16)Online publication date: 11-May-2024

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media