Berardina De Carolis
ABSTRACT
Children with Autism Spectrum Disorders (ASD) are characterized by impairments in communication and social skills, including problems in understanding and producing gestures. Using the approach of robot-based imitation games, in this paper, we propose the prototype of an imitation game that aims at improving the non-verbal communication skills, gestures in particular, of children with ASD. Starting from an application that we developed in another domain, social inclusion of migrant children, we use a social robot to teach them to recognize and produce social gestures through an imitation game. For allowing the recognition of gestures by the robot, we learned a LSTM-based model using MediaPipe for the analysis of hands positions and landmarks. The model was trained on six selected gestures for recognizing their pattern. The module is then used by the robot in the game. Results from the software accuracy point of view are encouraging and show that the proposed approach is suitable for the purpose of showing and recognizing predefined gestures, however we are aware that in the wild with ASD children it might not work properly. For this reason, in the near future, we will perform a study aiming at assessing the efficacy of the approach with ASD children and revise the model and the game accordingly.
- JB Leaf ad Leaf R., Mceachin J., M. Taubman, Shahla Ala’i-Rosales, R. K. Ross, Tristram H. Smith, and M. Weiss. 2016. ”Applied Behavior Analysis is a Science and, Therefore, Progressive”. ”Journal of Autism and Developmental Disorders” 46 (2016), 720–731.Google Scholar
- AP. Association. 1984. ”Diagnostic and statistical manual of mental disorders”. (1984).Google Scholar
- De Carolis B., D’Errico F., Macchiarulo N., and Palestra G.2021. ”Socially Inclusive Robots: Learning Culture-Related Gestures by Playing with Pepper”.Google Scholar
- De Carolis B., Palestra G., della Penna C., Cianciotta M., and Cervelione A.2019. ”Social Robots supporting the Inclusion of Unaccompanied Migrant Children: Teaching the Meaning of Culture-Related Gestures”. ”Journal of E-Learning and Knowledge Society 15” (2019).Google Scholar
- J. S. Bruner. 1981. The social context of language acquisition. Language & Communication(1981).Google Scholar
- So W C, Wong MK, Lam CK, Lam WY, Chui AT, Lee TL, Ng HM, Chan CH, and Fok DC. 2018. Using a social robot to teach gestural recognition and production in children with autism spectrum disorders. Disabil Rehabil Assist Technol 6 (08 2018), 527–539. https://doi.org/10.1080/17483107.2017.1344886Google Scholar
- Mauro Conti, Roberto Di Pietro, Luigi V. Mancini, and Alessandro Mei. 2009. (old) Distributed data source verification in wireless sensor networks. Inf. Fusion 10, 4 (2009), 342–353. https://doi.org/10.1016/j.inffus.2009.01.002Google ScholarDigital Library
- E. P. Bernier D. Leyzberg F. Shic R. Pauland B. Scassellati E. S. Kim, L. D. Berkovits. 2013. Social robots as embedded reinforcers of socialbehavior in children with autism. J. Autism Dev. Disord 43.Google Scholar
- Palestra G., De Carolis B., and Esposito F.2017. Artificial Intelligence for Robot-Assisted Treatment of Autism. In WAIAH@AI*IA.Google Scholar
- Palestra G., Varni G., Chetouani M., and Esposito F.2016. ”A multimodal and multilevel system for robotics treatment of autism in children”. 1–6. https://doi.org/10.1145/3005338.3005341Google Scholar
- Lalonde K. Ingersoll B. 2010. The impact of object and gesture imitation training on language use in children with autism spectrum disorder. J Speech Lang Hear Res. 53 (2010), 1040–1051.Google ScholarCross Ref
- A. Kendon. 1980. ”Gesticulation and speech: two aspects of the process of utterance”. ”In M. R. Key (ed), The Relationship of Verbal and Nonverbal Communication, pp. 207-227. The Hague Mouton and Co.”(1980).Google ScholarCross Ref
- Flavia Lecciso, Annalisa Levante, Rosa Angela Fabio, Tindara Caprì, Marco Leo, Pierluigi Carcagnì, Cosimo Distante, Pier Luigi Mazzeo, Paolo Spagnolo, and Serena Petrocchi. 2021. Emotional Expression in Children With ASD: A Pre-Study on a Two-Group Pre-Post-Test Design Comparing Robot-Based and Computer-Based Training. Frontiers in Psychology 12 (2021), 2826. https://doi.org/10.3389/fpsyg.2021.678052Google ScholarCross Ref
- P. Carcagni C. Distante M. Bernava G. Pioggiaand G. Palestra M. Leo, M. Del Coco. 2015. Automatic emotion recognition in robot–childreninteraction for ASD treatment. IEEE Int. Conf. ComputerVision Workshops.Google Scholar
- David McNeill. 2000. Introduction. Cambridge University Press, 1–10. https://doi.org/10.1017/CBO9780511620850.001Google Scholar
- Vallee L. N., Lohr C., Nguyen S. M., Kanellos I., and Asseu O.2019. BUILDING AN AUTOMATED GESTURE IMITATION GAME FOR TEENAGERS WITH ASD. ”Far East Journal of Electronics and Communications” (2019).Google Scholar
- Scassellati. 2007. How Social Robots Will Help Us to Diagnose Treat and Understand Autism. ”Thrun S., Brooks R., Durrant-Whyte H. (eds) Robotics Research. Springer Tracts in Advanced Robotics, vol 28. Springer, Berlin, Heidelberg” (2007).Google Scholar
- Alex Sherstinsky. 2020. Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Physica D: Nonlinear Phenomena 404 (2020), 132306.Google ScholarCross Ref
- Hiroki Tanaka, Hideki Negoro, Hidemi Iwasaka, and Satoshi Nakamura. 2017. Embodied conversational agents for multimodal automated social skills training in people with autism spectrum disorders. PLOS ONE 12 (08 2017), e0182151. https://doi.org/10.1371/journal.pone.0182151Google Scholar
- Carlos Valadão, Christiane Goulart, Hamilton Rivera, Eliete Caldeira, Teodiano Freire, Anselmo Frizera, and Ricardo Carelli. 2016. ”Analysis of the use of a robot to improve social skills in children with autism spectrum disorder”. Research on Biomedical Engineering 32 (06 2016), 161–175. https://doi.org/10.1590/2446-4740.01316Google Scholar
- Fangfang Zhang, Valentin Bazarevsky, Andrey Vakunov, A. Tkachenka, George Sung, Chuo-Ling Chang, and Matthias Grundmann. 2020. MediaPipe Hands: On-device Real-time Hand Tracking. ArXiv abs/2006.10214(2020).Google Scholar
Index Terms
- Social Robots to Support Gestural Development in Children with Autism Spectrum Disorder
Recommendations
Is the social robot probo an added value for social story intervention for children with autism spectrum disorders?
HRI '12: Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot InteractionIn this paper, we describe the first results of using the robot Probo as a facilitator in Social Story Intervention for children with autism spectrum disorders (ASD). Four preschoolers diagnosed with ASD participated in this research. For each of them, ...
Comments