Skip to main content
SpringerLink
Log in

Towards the Development of a Machine Learning-Based Action Recognition Model to Support Positive Behavioural Outcomes in Students with Autism

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1792))

Included in the following conference series:

  • 600 Accesses

Abstract

With the increasing prevalence of autism, it is imperative to develop new strategies and tools to help caregivers, parents and teaching staff support the needs of students with autism. In particular, children may experience highly stressful events, sometimes termed ‘meltdown’ or ‘emotional dysregulation’ events, which are preceded by a ‘rumble’ stage that could be detected and acted upon in a timely manner. Among the many possible solutions, the use of technology and, in particular, Artificial Intelligence is promising, thanks to the recent advancements in research. Our study focuses on the development of an action recognition model to detect and distinguish the six most common actions that children with autism exhibit during the rumble stage when approaching a meltdown. In doing so, we think caregivers, parents and teaching staff would be able to use the inferences generated by the model and intervene with evidence-based well-being practices to address such issues before escalation and decrease the frequency and intensity of such events.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adiba, F.I., et al.: Effect of corpora on classification of fake news using naive bayes classifier. Int. J. Autom. Artif. Intell. Mach. Learn. 1(1), 80–92 (2020)

    Google Scholar 

  2. Ahmed, S., Hossain, M.F., Nur, S.B., Shamim Kaiser, M., Mahmud, M.: Toward machine learning-based psychological assessment of autism spectrum disorders in school and community. In: Kaiser, M.S., Bandyopadhyay, A., Ray, K., Singh, R., Nagar, V. (eds.) Proceedings of Trends in Electronics and Health Informatics. LNNS, vol. 376, pp. 139–149. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-8826-3_13

    Chapter  Google Scholar 

  3. Ahmed, S., Hossain, M.F., Kaiser, M.S., Noor, M.B.T., Mahmud, M., Chakraborty, C.: Artificial intelligence and machine learning for ensuring security in smart cities. In: Chakraborty, C., Lin, J.C.-W., Alazab, M. (eds.) Data-Driven Mining, Learning and Analytics for Secured Smart Cities. ASTSA, pp. 23–47. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72139-8_2

    Chapter  Google Scholar 

  4. Niamat Ullah Akhund, T.M., Mahi, M.J.N., Hasnat Tanvir, A.N.M., Mahmud, M., Kaiser, M.S.: ADEPTNESS: Alzheimer’s disease patient management system using pervasive sensors - early prototype and preliminary results. In: Wang, S., et al. (eds.) BI 2018. LNCS (LNAI), vol. 11309, pp. 413–422. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-05587-5_39

    Chapter  Google Scholar 

  5. Al Banna, M.H., Ghosh, T., Taher, K.A., Kaiser, M.S., Mahmud, M.: A monitoring system for patients of autism spectrum disorder using artificial intelligence. In: Mahmud, M., Vassanelli, S., Kaiser, M.S., Zhong, N. (eds.) BI 2020. LNCS (LNAI), vol. 12241, pp. 251–262. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59277-6_23

    Chapter  Google Scholar 

  6. AlArjani, A., et al.: Application of mathematical modeling in prediction of COVID-19 transmission dynamics. Arab. J. Sci. Eng. 47, 10163–10186 (2022)

    Google Scholar 

  7. Albawi, S., Mohammed, T.A., Al-Zawi, S.: Understanding of a convolutional neural network. In: 2017 International Conference on Engineering and Technology (ICET), pp. 1–6. IEEE (2017)

    Google Scholar 

  8. Ali, A., Negin, F., Bremond, F., Thümmler, S.: Video-based behavior understanding of children for objective diagnosis of autism. In: VISAPP 2022-International Conference on Computer Vision Theory and Applications (2022)

    Google Scholar 

  9. Bhapkar, H.R., Mahalle, P.N., Shinde, G.R., Mahmud, M.: Rough sets in COVID-19 to predict symptomatic cases. In: Santosh, K.C., Joshi, A. (eds.) COVID-19: Prediction, Decision-Making, and its Impacts. LNDECT, vol. 60, pp. 57–68. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-9682-7_7

    Chapter  Google Scholar 

  10. Biswas, M., Kaiser, M.S., Mahmud, M., Al Mamun, S., Hossain, M.S., Rahman, M.A.: An XAI based autism detection: the context behind the detection. In: Mahmud, M., Kaiser, M.S., Vassanelli, S., Dai, Q., Zhong, N. (eds.) BI 2021. LNCS (LNAI), vol. 12960, pp. 448–459. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86993-9_40

    Chapter  Google Scholar 

  11. Biswas, M., et al.: ACCU3RATE: a mobile health application rating scale based on user reviews. PLoS One 16(12), e0258050 (2021)

    Google Scholar 

  12. Biswas, M., Rahman, A., Kaiser, M.S., Al Mamun, S., Ebne Mizan, K.S., Islam, M.S., Mahmud, M.: Indoor navigation support system for patients with neurodegenerative diseases. In: Mahmud, M., Kaiser, M.S., Vassanelli, S., Dai, Q., Zhong, N. (eds.) BI 2021. LNCS (LNAI), vol. 12960, pp. 411–422. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86993-9_37

    Chapter  Google Scholar 

  13. Del Coco, M., et al.: A computer vision based approach for understanding emotional involvements in children with autism spectrum disorders. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1401–1407 (2017)

    Google Scholar 

  14. Di Nuovo, A., Conti, D., Trubia, G., Buono, S., Di Nuovo, S.: Deep learning systems for estimating visual attention in robot-assisted therapy of children with autism and intellectual disability. Robotics 7(2), 25 (2018)

    Article  Google Scholar 

  15. Farhin, F., Kaiser, M.S., Mahmud, M.: Towards secured service provisioning for the internet of healthcare things. In: Proceedings of the AICT, pp. 1–6 (2020)

    Google Scholar 

  16. Farhin, F., Kaiser, M.S., Mahmud, M.: Secured smart healthcare system: blockchain and Bayesian inference based approach. In: Kaiser, M.S., Bandyopadhyay, A., Mahmud, M., Ray, K. (eds.) Proceedings of International Conference on Trends in Computational and Cognitive Engineering. AISC, vol. 1309, pp. 455–465. Springer, Singapore (2021). https://doi.org/10.1007/978-981-33-4673-4_36

    Chapter  Google Scholar 

  17. Ghosh, T., et al.: Artificial intelligence and internet of things in screening and management of autism spectrum disorder. Sustain. Cities Soc. 74, 103189 (2021)

    Google Scholar 

  18. Google: Mediapipe. https://mediapipe.dev. Accessed 30 July 2022

  19. Google: Mediapipe pipeline. https://google.github.io/mediapipe/solutions/holistic. Accessed 30 July 2022

  20. Heinsfeld, A.S., Franco, A.R., Craddock, R.C., Buchweitz, A., Meneguzzi, F.: Identification of autism spectrum disorder using deep learning and the abide dataset. NeuroImage Clin. 17, 16–23 (2018)

    Article  Google Scholar 

  21. Islam, N., et al.: Towards machine learning based intrusion detection in IoT networks. Comput. Mater. Contin. 69(2), 1801–1821 (2021)

    Google Scholar 

  22. Jarraya, S.K., Masmoudi, M., Hammami, M.: Compound emotion recognition of autistic children during meltdown crisis based on deep spatio-temporal analysis of facial geometric features. IEEE Access 8, 69311–69326 (2020)

    Article  Google Scholar 

  23. Jesmin, S., Kaiser, M.S., Mahmud, M.: Artificial and internet of healthcare things based Alzheimer care during COVID 19. In: Mahmud, M., Vassanelli, S., Kaiser, M.S., Zhong, N. (eds.) BI 2020. LNCS (LNAI), vol. 12241, pp. 263–274. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59277-6_24

    Chapter  Google Scholar 

  24. Jiao, Y., Lu, Z.: Predictive models for autism spectrum disorder based on multiple cortical features. In: 2011 Eighth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), vol. 3, pp. 1611–1615. IEEE (2011)

    Google Scholar 

  25. Kaiser, M.S., et al.: 6G access network for intelligent internet of healthcare things: opportunity, challenges, and research directions. In: Kaiser, M.S., Bandyopadhyay, A., Mahmud, M., Ray, K. (eds.) Proceedings of International Conference on Trends in Computational and Cognitive Engineering. AISC, vol. 1309, pp. 317–328. Springer, Singapore (2021). https://doi.org/10.1007/978-981-33-4673-4_25

    Chapter  Google Scholar 

  26. Karim, S., Akter, N., Patwary, M.J., Islam, M.R.: A review on predicting autism spectrum disorder (ASD) meltdown using machine learning algorithms. In: 2021 5th International Conference on Electrical Engineering and Information & Communication Technology (ICEEICT), pp. 1–6. IEEE (2021)

    Google Scholar 

  27. Kumar, S., et al.: Forecasting major impacts of COVID-19 pandemic on country-driven sectors: challenges, lessons, and future roadmap. Pers. Ubiquit. Comput. 1–24 (2021)

    Google Scholar 

  28. Landowska, A., et al.: Automatic emotion recognition in children with autism: a systematic literature review. Sensors 22(4), 1649 (2022)

    Article  Google Scholar 

  29. Liu, W., Yu, X., Raj, B., Yi, L., Zou, X., Li, M.: Efficient autism spectrum disorder prediction with eye movement: A machine learning framework. In: 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 649–655. IEEE (2015)

    Google Scholar 

  30. Lord, C., Elsabbagh, M., Baird, G., Veenstra-Vanderweele, J.: Autism spectrum disorder. Lancet 392(10146), 508–520 (2018)

    Article  Google Scholar 

  31. Mahmud, M., Kaiser, M.S.: Machine learning in fighting pandemics: a COVID-19 case study. In: Santosh, K.C., Joshi, A. (eds.) COVID-19: Prediction, Decision-Making, and its Impacts. LNDECT, vol. 60, pp. 77–81. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-9682-7_9

    Chapter  Google Scholar 

  32. Mahmud, M., Kaiser, M.S., McGinnity, T.M., Hussain, A.: Deep learning in mining biological data. Cogn. Comput. 13(1), 1–33 (2021)

    Article  Google Scholar 

  33. Mahmud, M., Kaiser, M.S., Hussain, A., Vassanelli, S.: Applications of deep learning and reinforcement learning to biological data. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2063–2079 (2018)

    Article  MathSciNet  Google Scholar 

  34. Mahmud, M., et al.: Towards explainable and privacy-preserving artificial intelligence for personalisation in autism spectrum disorder. In: Antona, M., Stephanidis, C. (eds.) HCII 2022. LNCS, vol. 13309, pp. 356–370. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-05039-8_26

    Chapter  Google Scholar 

  35. Masmoudi, M., Jarraya, S.K., Hammami, M.: MeltdownCrisis: dataset of autistic children during meltdown crisis. In: 2019 15th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), pp. 239–246. IEEE (2019)

    Google Scholar 

  36. Myles, B.S., Hubbard, A.: The cycle of tantrums, rage, and meltdowns in children and youth with asperger syndrome, high-functioning autism, and related disabilities. In: CDROM ISEC 2005 Inclusive and Supportive Education Congress, vol. 10, p. 05 (2005). www.inclusive.co.uk

  37. Nahiduzzaman, M., Tasnim, M., Newaz, N.T., Kaiser, M.S., Mahmud, M.: Machine learning based early fall detection for elderly people with neurological disorder using multimodal data fusion. In: Mahmud, M., Vassanelli, S., Kaiser, M.S., Zhong, N. (eds.) BI 2020. LNCS (LNAI), vol. 12241, pp. 204–214. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59277-6_19

    Chapter  Google Scholar 

  38. Nawar, A., et al.: Cross-content recommendation between movie and book using machine learning. In: Proceedings of the AICT, pp. 1–6 (2021)

    Google Scholar 

  39. Noor, M.B.T., Zenia, N.Z., Kaiser, M.S., Mamun, S.A., Mahmud, M.: Application of deep learning in detecting neurological disorders from magnetic resonance images: a survey on the detection of Alzheimer’s disease, Parkinson’s disease and schizophrenia. Brain Inform. 7(1), 1–21 (2020)

    Article  Google Scholar 

  40. Patnam, V.S.P., George, F.T., George, K., Verma, A.: Deep learning based recognition of meltdown in autistic kids. In: 2017 IEEE International Conference on Healthcare Informatics (ICHI), pp. 391–396. IEEE (2017)

    Google Scholar 

  41. Paul, A., et al.: Inverted bell-curve-based ensemble of deep learning models for detection of COVID-19 from chest x-rays. Neural Comput. Appl. 1–15 (2022)

    Google Scholar 

  42. Prakash, N., et al.: Deep transfer learning for COVID-19 detection and infection localization with superpixel based segmentation. Sustain. Cities Soc. 75, 103252 (2021)

    Google Scholar 

  43. Rabby, G., Azad, S., Mahmud, M., Zamli, K.Z., Rahman, M.M.: TeKET: a tree-based unsupervised keyphrase extraction technique. Cogn. Comput. 12(4), 811–833 (2020)

    Article  Google Scholar 

  44. Rabby, G., et al.: A flexible keyphrase extraction technique for academic literature. Procedia Comput. Sci. 135, 553–563 (2018)

    Article  Google Scholar 

  45. Rahman, M.A., Brown, D.J., Shopland, N., Burton, A., Mahmud, M.: Explainable multimodal machine learning for engagement analysis by continuous performance test. In: Antona, M., Stephanidis, C. (eds.) HCII 2022. LNCS, vol. 13309, pp. 386–399. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-05039-8_28

    Chapter  Google Scholar 

  46. Rahman, M.A., et al.: Towards machine learning driven self-guided virtual reality exposure therapy based on arousal state detection from multimodal data. In: Mahmud, M., He, J., Vassanelli, S., van Zundert, A., Zhong, N. (eds.) BI 2022. L, vol. 13406, pp. 195–209. Springer, Cham (2022)

    Chapter  Google Scholar 

  47. Rudovic, O., et al.: CultureNet: a deep learning approach for engagement intensity estimation from face images of children with autism. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 339–346. IEEE (2018)

    Google Scholar 

  48. Satu, M.S., et al.: Short-term prediction of COVID-19 cases using machine learning models. Appl. Sci. 11(9), 4266 (2021)

    Article  Google Scholar 

  49. Society, N.A.: Meltdowns - a guide for all audiences. https://www.autism.org.uk/advice-and-guidance/topics/behaviour/meltdowns/all-audiences

  50. Sumi, A.I., Zohora, M.F., Mahjabeen, M., Faria, T.J., Mahmud, M., Kaiser, M.S.: fASSERT: a fuzzy assistive system for children with autism using internet of things. In: Wang, S., et al. (eds.) BI 2018. LNCS (LNAI), vol. 11309, pp. 403–412. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-05587-5_38

  51. Tamilarasi, F.C., Shanmugam, J.: Convolutional neural network based autism classification. In: 2020 5th International Conference on Communication and Electronics Systems (ICCES), pp. 1208–1212. IEEE (2020)

    Google Scholar 

  52. Zaman, S., et al.: Security threats and artificial intelligence based countermeasures for internet of things networks: a comprehensive survey. IEEE Access 9, 94668–94690 (2021)

    Article  Google Scholar 

  53. Zhou, Y., Yu, F., Duong, T.: Multiparametric MRI characterization and prediction in autism spectrum disorder using graph theory and machine learning. PLoS One 9(6), e90405 (2014)

    Google Scholar 

Download references

Acknowledgement

This work is supported by the AI-TOP (2020-1-UK01-KA201-079167) and DIVERSASIA (618615-EPP-1-2020-1-UKEPPKA2-CBHEJP) projects funded by the European Commission under the Erasmus+ programme.

Author information

Authors and Affiliations

  1. Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK

    Francesco Bonacini, Mufti Mahmud & David J. Brown

  2. Computing and Informatics Research Centre, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK

    Francesco Bonacini, Mufti Mahmud & David J. Brown

  3. Medical Technologies Innovation Facility, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK

    Mufti Mahmud & David J. Brown

Authors
  1. Francesco Bonacini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mufti Mahmud
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David J. Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mufti Mahmud .

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Indore, Indore, India

    Mohammad Tanveer

  2. Indian Institute of Information Technology - Allahabad, Prayagraj, India

    Sonali Agarwal

  3. Kobe University, Kobe, Japan

    Seiichi Ozawa

  4. Indian Institute of Technology Patna, Patna, India

    Asif Ekbal

  5. University of Innsbruck, Innsbruck, Austria

    Adam Jatowt

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bonacini, F., Mahmud, M., Brown, D.J. (2023). Towards the Development of a Machine Learning-Based Action Recognition Model to Support Positive Behavioural Outcomes in Students with Autism. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Communications in Computer and Information Science, vol 1792. Springer, Singapore. https://doi.org/10.1007/978-981-99-1642-9_50

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-1642-9_50

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-1641-2

  • Online ISBN: 978-981-99-1642-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation