Critical Analysis on the Design and Development of an E-Learning System for Autism Spectrum Disorder Intervention through Gamified Augmented Reality and Emotion-Recognition
Pages 400 - 409
Abstract
In the Philippines the state of care for those with Autism Spectrum Disorder (ASD) is affected by factors such as upfront cost, the availability of care staff and care resources, and the limited integration of technology. Individuals with ASD often have trouble reliably identifying emotions in social situations, a known trait that has inspired much research into relevant assistive technologies. Many technologies have been proposed to aid in improving the condition of those with ASD via various means. This paper discusses a system which aims to use computer vision integrated with Augmented Reality (AR) to develop a novel assistive application. This application's purpose is to aid care professionals working with individuals with ASD through a system that can recognize facial expressions, output a reading on said emotion, and then display that output in a real-time AR display.
References
[1]
Syriopoulou-Delli, C.K. and Gkiolnta, E., 2022. Review of assistive technology in the training of children with autism spectrum disorders. International Journal of Developmental Disabilities, 68(2), pp.73-85. https://doi.org/10.1080/20473869.2019.1706333
[2]
Hodges, H., Fealko, C. and Soares, N., 2020. Autism spectrum disorder: definition, epidemiology, causes, and clinical evaluation. Translational pediatrics, 9(Suppl 1), p.S55. https://doi.org/10.21037/tp.2019.09.09.
[3]
Chapman, R., 2020. The reality of autism: On the metaphysics of disorder and diversity. Philosophical Psychology, 33(6), pp.799-819. https://doi.org/10.1080/09515089.2020.1751103
[4]
Rauschnabel, P.A., Babin, B.J., Dieck, M.C., Krey, N. and Jung, T., 2022. What is augmented reality marketing? Its definition, complexity, and future. Journal of business research, 142, pp.1140-1150. https://doi.org/10.1016/j.jbusres.2021.12.084
[5]
Abdullah, S.M.S.A., Ameen, S.Y.A., Sadeeq, M.A. and Zeebaree, S., 2021. Multimodal emotion recognition using deep learning. Journal of Applied Science and Technology Trends, 2(01), pp.73-79, https://doi.org/10.38094/jastt20291.
[6]
Sarvakar, K., Senkamalavalli, R., Raghavendra, S., Kumar, J.S., Manjunath, R. and Jaiswal, S., 2023. Facial emotion recognition using convolutional neural networks. Materials Today: Proceedings, 80, pp.3560-3564, https://doi.org/10.1016/j.matpr.2021.07.297.
[7]
Quilendrino, M.I.O., Castor, M.A.R., Mendoza, N.R.N.P., Vea, J.R. and Castillo-Carandang, N.T., 2022. The Direct Cost of Autism and its Economic Impact on the Filipino Family. Acta Medica Philippina, 56(9), https://doi.org/10.47895/amp.v56i9.5005.
[8]
Prakash, V.G., Kohli, M., Kohli, S., Prathosh, A.P., Wadhera, T., Das, D., Panigrahi, D. and Kommu, J.V.S., 2023. Computer vision-based assessment of autistic children: Analyzing interactions, emotions, human pose, and life skills. IEEE Access, 11, pp.47907-47929, https://doi.org/10.1109/ACCESS.2023.3269027.
[9]
Manju, T., Magesh, Padmavathi, S. and Durairaj, 2023. Increasing the Social Interaction of Autism Child Using Virtual Reality Intervention (VRI). ACM Transactions on Asian and Low-Resource Language Information Processing, https://doi.org/10.1145/3592855.
[10]
The World Bank, 2024. World Bank Country and Lending Groups, Country Classification.
[11]
Hatta, M.H., Sidi, H., Siew Koon, C., Che Roos, N.A., Sharip, S., Abdul Samad, F.D., Wan Xi, O., Das, S. and Mohamed Saini, S., 2022. Virtual reality (VR) technology for treatment of mental health problems during COVID-19: a systematic review. International Journal of Environmental Research and Public Health, 19(9), p.5389. https://doi.org/ 10.3390/ijerph19095389
[12]
Karami, B., Koushki, R., Arabgol, F., Rahmani, M. and Vahabie, A.H., 2021. Effectiveness of virtual/augmented reality–based therapeutic interventions on individuals with autism spectrum disorder: a comprehensive meta-analysis. Frontiers in Psychiatry, 12, p.665326., https://doi.org/ 10.3389/fpsyt.2021.665326.
[13]
Berenguer, C., Baixauli, I., Gómez, S., Andrés, M.D.E.P. and De Stasio, S., 2020. Exploring the impact of augmented reality in children and adolescents with autism spectrum disorder: A systematic review. International Journal of Environmental Research and Public Health, 17(17), p.6143., https://doi.org/ 10.3390/ijerph17176143.
[14]
Haque, M.I.U. and Valles, D., 2019, October. Facial expression recognition using dcnn and development of an ios app for children with asd to enhance communication abilities. In 2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) (pp. 0476-0482). IEEE, https://doi.org/ 10.1109/CSCI46756.2018.00090.
[15]
Drimalla, H., Baskow, I., Behnia, B., Roepke, S. and Dziobek, I., 2021. Imitation and recognition of facial emotions in autism: a computer vision approach. Molecular autism, 12, pp.1-15, https://doi.org/10.1186/s13229-021-00430-0.
[16]
Donohue, M.R., Childs, A.W., Richards, M. and Robins, D.L., 2019. Race influences parent report of concerns about symptoms of autism spectrum disorder. Autism, 23(1), pp.100-111., https://doi.org/ 10.1177/1362361317722030
[17]
Zang, X., Liu, L., Wang, M., Song, Y., Zhang, H. and Chen, J., 2021. Photochat: A human-human dialogue dataset with photo sharing behavior for joint image-text modeling. arXiv preprint arXiv:2108.01453.
[18]
Kumar, S., Devi, B. and Goud, V., 2019. A comparative study of acne vulgaris with special reference to therapeutic options. IP Indian J. Clin. Exp. Dermatol, 5, pp.306-311.
[19]
Ahmad, H.B., 2019. An augmented reality system to enhance facial expressions recognision in autistic children. Journal of Advanced Computer Science & Technology, 8(2), pp.46-49., https://doi.org/10.14419/jacst.v8i2.29873.
[20]
Zhang, M., Ding, H., Naumceska, M. and Zhang, Y., 2022. Virtual reality technology as an educational and intervention tool for children with autism spectrum disorder: current perspectives and future directions. Behavioral Sciences, 12(5), p.138, https://doi.org/10.3390/bs12050138
[21]
Lee, J.R.H. and Wong, A., 2020. AEGIS: A real-time multimodal augmented reality computer vision based system to assist facial expression recognition for individuals with autism spectrum disorder. arXiv preprint arXiv:2010.11884, https://doi.org/10.48550/arXiv.2010.11884.
[22]
Lee, J.R.H. and Wong, A., 2020. Timeconvnets: A deep time windowed convolution neural network design for real-time video facial expression recognition. In 2020 17th Conference on Computer and Robot Vision (CRV) (pp. 9-16). IEEE, https://doi.org/10.48550/arXiv.2003.01791
[23]
Prakash, K.B. and Kanagachidambaresan, G.R., 2021. Programming with TensorFlow. EAI/Springer Innovations in Communication and Computing.
[24]
Balakrishnan, S., Hameed, M.S.S., Venkatesan, K. and Aswin, G., 2021, March. Interaction of spatial computing in augmented reality. In 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS) (Vol. 1, pp. 1900-1904). IEEE, https://doi.org/ 10.1109/ICACCS51430.2021.9442010
[25]
Culot, G., Nassimbeni, G., Podrecca, M. and Sartor, M., 2021. The ISO/IEC 27001 information security management standard: literature review and theory-based research agenda. The TQM Journal, 33(7), pp.76-105, https://doi.org/10.1108/TQM-09-2020-0202.
[26]
Shahid, J., Ahmad, R., Kiani, A.K., Ahmad, T., Saeed, S. and Almuhaideb, A.M., 2022. Data protection and privacy of the internet of healthcare things (IoHTs). Applied Sciences, 12(4), p.1927., https://doi.org/10.3390/app12041927
[27]
Penev, Y., Dunlap, K., Husic, A., Hou, C., Washington, P., Leblanc, E., Kline, A., Kent, J., Ng-Thow-Hing, A., Liu, B. and Harjadi, C., 2021. A mobile game platform for improving social communication in children with autism: a feasibility study. Applied clinical informatics, 12(05), pp.1030-1040., https://doi.org/10.1055/s-0041-1736626.
[28]
Boboc, R.G., Chiriac, R.L. and Antonya, C., 2021. How augmented reality could improve the student's attraction to learn mechanisms. Electronics, 10(2), p.175., https://doi.org/10.3390/electronics10020175
[29]
Hajesmaeel-Gohari, S. and Bahaadinbeigy, K., 2021. The most used questionnaires for evaluating telemedicine services. BMC medical informatics and decision making, 21(1), pp.1-11.
[30]
Almurashi, H., Bouaziz, R., Alharthi, W., Al-Sarem, M., Hadwan, M. and Kammoun, S., 2022. Augmented reality, serious games and picture exchange communication system for people with ASD: systematic literature review and future directions. Sensors, 22(3), p.1250., https://doi.org/ 10.3390/s22031250
[31]
Silva, S.D., de Sousa Neto, A.F., Neto, F.M.M., de Lima, R.M. and do Amaral Demoly, K.R., 2020, November. A Serious Game with Geolocalition to Support Learning ofChildren with Autism and Learning Difficulties. In Simpósio Brasileiro de Informática na Educação (SBIE) (pp. 342-351). SBC, https://doi.org/10.5753/cbie.sbie.2020.342.
[32]
Tsikinas, S. and Xinogalos, S., 2020. Towards a serious games design framework for people with intellectual disability or autism spectrum disorder. Education and Information Technologies, 25(4), pp.3405-3423., https://doi.org/10.1007/s10639-020-10124-4.
[33]
Papoutsi, C., Drigas, A. and Skianis, C., 2021. Virtual and augmented reality for developing emotional intelligence skills. Int. J. Recent Contrib. Eng. Sci. IT (IJES), 9(3), pp.35-53., https://doi.org/10.3991/ijes.v9i3.23939.
[34]
Jaramillo-Alcázar, A., Arias, J., Albornoz, I., Alvarado, A. and Luján-Mora, S., 2022. Method for the development of accessible mobile serious games for children with autism spectrum disorder. International Journal of Environmental Research and Public Health, 19(7), p.3844, https://doi.org/10.3390/ijerph19073844.
[35]
de Carvalho, A.P., Braz, C.S., dos Santos, S.M., Ferreira, R.A. and Prates, R.O., 2024. Serious games for children with autism spectrum disorder: A systematic literature review. International Journal of Human–Computer Interaction, 40(14), pp.3655-3682, https://doi.org/10.1080/10447318.2023.2194051
[36]
Paredes-Xochihua, M.P., 2023. Analysis of mobile applications for adults with autism spectrum disorder. Journal Computer Technology, 7(19).
[37]
Sweidan, S.Z., Salameh, H., Zakarneh, R. and Darabkh, K.A., 2022. Autistic Innovative Assistant (AIA): an Android application for Arabic autism children. Interactive Learning Environments, 30(4), pp.735-758, https://doi.org/10.1080/10494820.2019.1681468
[38]
Ozsahin, I., Mustapha, M.T., Albarwary, S., Sanlidag, B., Ozsahin, D.U. and Butler, T.A., 2021. An investigation to choose the proper therapy technique in the management of autism spectrum disorder. Journal of Comparative Effectiveness Research, 10(5), pp.423-437, https://doi.org/10.2217/cer-2020-0162
[39]
Nezamuldeen, L. and Jafri, M.S., 2024. Text Mining to Understand Disease-Causing Gene Variants. Knowledge, 4(3), pp.422-443.
[40]
Popoola, O.A., Adama, H.E., Okeke, C.D. and Akinoso, A.E., 2024. Conceptualizing agile development in digital transformations: Theoretical foundations and practical applications. Engineering Science & Technology Journal, 5(4), pp.1524-1541.
[41]
Fagarasan, C., Popa, O., Pisla, A. and Cristea, C., 2021. Agile, waterfall and iterative approach in information technology projects. In IOP Conference Series: Materials Science and Engineering (Vol. 1169, No. 1, p. 012025). IOP Publishing.
[42]
Mohajan, H.K., 2020. Quantitative research: A successful investigation in natural and social sciences. Journal of Economic Development, Environment and People, 9(4), pp.50-79.
[43]
Jian, T., 2022. Research on the design of intelligent virtual simulation experimental teaching platform based on augmented reality. International Journal of New Developments in Engineering and Society, 6(4), https://doi.org/10.25236/IJNDES.2022.060405.
[44]
Chistol, M., Turcu, C. and Danubianu, M., 2023. Autism assistant: A platform for autism home-based therapeutic intervention. IEEE Access.
[45]
Rahmawati, E.F., Ayuningtyas, A. and Sagirani, T., 2022. Penerapan Metode Double Diamond pada Desain User Interface Website: The Implementation of the Double Diamond Method on the Design User Interface Website. Jurnal Komunika: Jurnal Komunikasi, Media dan Informatika, 11(1), pp.11-22.
[46]
Gilabert-Cerdá, A., Lledó, G.L., Lorenzo-Lledó, A., Carreres, A.L. and Pérez-Vázquez, E., 2023. Augmented Reality Promotes Social Responses In Autism Spectrum Disorder Subjects, https://doi.org/10.36315/2023v2end096.
[47]
Kelly, C. and Kelly, C., 2021. What is different about eating disorders for those with autistic spectrum condition. Int J Psychiatry Res, 4(2), pp.1-8.
[48]
Ciray, R.O. and Turan, S., 2024. Searching the underlying mechanisms of specific learning disorder: An emotion recognition and social cognition aspect for Turkish clinical youth population. Applied Neuropsychology: Child, 13(3), pp.239-245.
[49]
Wilhelm, O., Hildebrandt, A., Manske, K., Schacht, A. and Sommer, W., 2014. Test battery for measuring the perception and recognition of facial expressions of emotion. Frontiers in psychology, 5, p.404, https://doi.org/10.3389/fpsyg.2014.00404.
[50]
Canoy, A., LPTa, J.C.B. and LPTb, W.E.C., 2022. The Lived Experiences of Teachers and Parents Having Children with Autism Spectrum Disorder During The Pandemic. The Lived Experiences of Teachers and Parents Having Children with Autism Spectrum Disorder During The Pandemic, 115(1), pp.17-17.
[51]
Gefenas, E., Lekstutiene, J., Lukaseviciene, V., Hartlev, M., Mourby, M. and Cathaoir, K.Ó., 2022. Controversies between regulations of research ethics and protection of personal data: informed consent at a cross-road. Medicine, Health Care and Philosophy, pp.1-8, https://doi.org/10.1007/s11019-021-10060-1
Index Terms
- Critical Analysis on the Design and Development of an E-Learning System for Autism Spectrum Disorder Intervention through Gamified Augmented Reality and Emotion-Recognition
Recommendations
An Avatar-Based Intervention System for Children with Autism Spectrum Disorder
Pattern Recognition and Computer VisionAbstractThe defects in the mirror neuron system (MNS) make it challenging for children with autism spectrum disorder (ASD) to learn social skills through third-person digital agents. To help children with ASD circumvent deficits in MNS, we propose an ...
Exploiting Augmented Reality in LEGO Therapy for Children with Autism Spectrum Disorder
Virtual Reality and Mixed RealityAbstractNumerous computer-based therapies have been designed for cognitive-behavioral interventions to support children with Autism Spectrum Disorder (ASD) in recent years. Among these technologies, Augmented Reality (AR) offers unique educational ...
Improved socio-emotional skills in students with autism spectrum disorder (ASD) following an intervention supported by an augmented gamified environment
AbstractThis study examines the impact of an intervention using an Augmented Gamified Environment (AGE) to stimulate socio-emotional skills in a sample of 54 subjects with Autism Spectrum Disorder (ASD) aged between 3 and 17 years old. The study used a ...
Highlights- Augmented Gamified Environment allows the stimulation of socio-emotional skills in students with ASD.
- Improvement in socio-emotional skills across all subjects post-intervention, irrespective of gender, age, severity of ASD, ...
Comments
Information & Contributors
Information
Published In
August 2024
465 pages
ISBN:9798400716997
DOI:10.1145/3695652
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 15 October 2024
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
IMMS 2024
IMMS 2024: 2024 the 7th International Conference on Information Management and Management Science
August 23 - 25, 2024
Beijing, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 52Total Downloads
- Downloads (Last 12 months)52
- Downloads (Last 6 weeks)19
Reflects downloads up to 08 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format