Abstract
Autism spectrum disorder (ASD) is a neurodevelopment disorders, which is very difficult to diagnose at early stages of life, and is a complex spectrum disorder, involving many persistent challenges with lack of social interactions, eye contact and facial expressions. Facial features of children having ASD are different from typically developing (TD) children, which is a major clue for early detection of autism using demographics of facial features. In this paper, a novel framework for early detection of autism has been developed, using a semi-supervised approach that consists ConvNeXt-T model and clustering of image embedding vectors. State-of-the-art ConvNeXt-T model is trained for autism dataset, and after that, image embedding clusters are created based on cosine similarity threshold. By doing this, model’s classification layer and embedding generation layers are trained for the subject data. The proposed framework provides embeddings and autism classifier from the trained model, where cluster embeddings can be used in semi-supervised classification. Hence, after training process, inference architecture has two modes: (i) classification using ConvNeXt-T and (ii) classification using embedding clusters. The classification accuracy of framework using ConvNeXt-T is 89.5%, where clustering accuracy using cosine similarity of image embedding is 93%. Most of the other techniques in this area focus only on classification using deep neural networks. The proposed framework is also providing classification based on embedding clusters and can work on unlabelled dataset. Moreover, the proposed framework is based on server–client architecture, which allows easy usability and integration without multi acceleration devices and extra resource consumption. A single image of child is required to detect autism using the proposed framework in real-life scenario.
Similar content being viewed by others
Data availability
Data shall be made available on request.
References
van’t Hof M, Tisseur C, van Berckelear-Onnes I, van Nieuwenhuyzen A, Daniels AM, Deen M, Ester WA (2021) Age at autism spectrum disorder diagnosis: a systematic review and meta-analysis from 2012 to 2019. Autism 25(4):862–873
Wergeland GJH, Posserud MB, Fjermestad K, Njardvik U, Öst LG (2022) Early behavioral interventions for children and adolescents with autism spectrum disorder in routine clinical care: a systematic review and meta-analysis. Clin Psychol Sci Pract 29(4):400
Hyman SL, Levy SE, Myers SM, Kuo DZ, Apkon S, Davidson LF, Bridgemohan C (2020) Identification, evaluation, and management of children with autism spectrum disorder. Pediatrics. https://doi.org/10.1542/peds.2019-3447
Goedeke S, Shepherd D, Landon J, Taylor S (2019) How perceived support relates to child autism symptoms and care-related stress in parents caring for a child with autism. Res Autism Spectr Disorders 60:36–47
Chen MH, Huang CF, Lin YS, Chiu YN, Gau SSF, Wu YY (2023) Validation of the mandarin chinese version of the autism diagnostic observation schedule-2 for autism spectrum disorder. Res Autism Spectr Disorders 105:102184
Raj S, Masood S (2020) Analysis and detection of autism spectrum disorder using machine learning techniques. Proc Comput Sci 167:994–1004
Nogay HS, Adeli H (2020) Machine learning (ML) for the diagnosis of autism spectrum disorder (ASD) using brain imaging. Rev Neurosci 31(8):825–841
Tan DW, Gilani SZ, Maybery MT, Mian A, Hunt A, Walters M, Whitehouse AJ (2017) Hypermasculinised facial morphology in boys and girls with autism spectrum disorder and its association with symptomatology. Sci Rep 7(1):9348
Boutrus M, Gilani SZ, Alvares GA, Maybery MT, Tan DW, Mian A, Whitehouse AJ (2019) Increased facial asymmetry in autism spectrum conditions is associated with symptom presentation. Autism Res 12(12):1774–1783
Mahapatra P, Pati S, Sinha R, Chauhan AS, Nanda RR, Nallala S (2019) Parental care-seeking pathway and challenges for autistic spectrum disorders children: a mixed method study from Bhubaneswar. Odisha Indian J Psychiatr 61(1):37
Ismail NAS, Ramli NS, Hamzaid NH, Hassan NI (2020) Exploring eating and nutritional challenges for children with autism spectrum disorder: parents’ and special educators’ perceptions. Nutrients 12(9):2530
Anagnostopoulou P, Alexandropoulou V, Lorentzou G, Lykothanasi A, Ntaountaki P, Drigas A (2020) Artificial intelligence in autism assessment. Int J Emerg Technol Learn (iJET) 15(6):95–107
de Belen RAJ, Bednarz T, Sowmya A, Del Favero D (2020) Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019. Transl Psychiatr 10(1):333
Abdelrahman M, Ali A, Farag A, Casanova MF, Farag A (2012) New approach for classification of autistic vs. typically developing brain using white matter volumes. In: Proc. Ninth Conference on Computer and Robot Vision, pp 284–289
Eslami T, Saeed F (2015) Auto-ASD-Network: A Technique Based on Deep Learning and Support Vector Machines for Diagnosing Autism Spectrum Disorder Using fMRI data. In: Proc 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, pp 646–651 (Association for Computing Machinery)
Kalantarian H et al (2018) A Gamified Mobile System for Crowdsourcing Video for Autism Research. In: Proc IEEE International Conference on Healthcare Informatics (ICHI), pp 350–352
Guha T, Yang Z, Grossman RB, Narayanan SS (2018) A computational study of expressive facial dynamics in children with autism. IEEE Trans Affect Comput 9:14–20
Shi L et al (2015) Different visual preference patterns in response to simple and complex dynamic social stimuli in preschool-aged children with autism spectrum disorders. PLoS ONE 10:e0122280
Chong E et al (2017) Detecting gaze towards eyes in natural social interactions and its use in child assessment. Proc ACM Interact Mob Wearable Ubiquitous Technol. https://doi.org/10.1145/3131902
Dawson G et al (2018) Atypical postural control can be detected via computer vision analysis in toddlers with autism spectrum disorder. Sci Rep 8:17008
Zunino A et al (2018) Video Gesture Analysis for Autism Spectrum Disorder Detection In: Proc 24th International Conference on Pattern Recognition (ICPR), pp 3421–3426
https://drive.google.com/drive/folders/1XQU0pluL0m3TIlXqntano12d68peMb8A
Shukla P, Gupta T, Saini A, Singh P, Balasubramanian R (2017) A Deep Learning Frame-Work for Recognizing Developmental Disorders. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, pp 705–714
Haque MIU, Valles D.(2018) A Facial Expression Recognition Approach Using DCNN for Autistic Children to Identify Emotions. In: 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), IEEE, pp 546–551
Rudovic O, Utsumi Y, Lee J, Hernandez J, Ferrer EC, Schuller B, Picard RW (2018). Culturenet: A Deep Learning Approach for Engagement Intensity Estimation from Face Images of Children with Autism. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, pp 339–346
Dong S, Wang P, Abbas K (2021) A survey on deep learning and its applications. Comput Sci Rev 40:100379
Liu Z, Mao H, Wu CY, Feichtenhofer C, Darrell T, Xie S (2022) A Convnet for the 2020s. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11976–11986
Trinh TH, Luong MT, Le QV (2019) Selfie: Self-Supervised Pretraining for image Embedding. arXiv preprint arXiv:1906.02940
Park K, Hong JS, Kim W (2020) A methodology combining cosine similarity with classifier for text classification. Appl Artif Intell 34(5):396–411
Yang E, Zhou W, Qian X, Yu L (2022) MGCNet: multilevel gated collaborative network for RGB-D semantic segmentation of indoorscene. IEEE Signal Process Lett 29:2567–2571
Haque MIU, Valles D (2018) A Facial Expression Recognition Approach Using DCNN for Autistic Children to Identify Emotions. In: 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), IEEE, pp 546–551
Jahanara S, Padmanabhan S (2021) Detecting autism from facial image. Int J Adv Res, Ideas Innov Technol 7(2):219–225
Alsaade FW, Alzahrani MS (2022) Classification and detection of autism spectrum disorder based on deep learning algorithms. Comput Intell Neurosci. https://doi.org/10.1155/2022/8709145
Akter T, Ali MH, Khan MI, Satu MS, Uddin MJ, Alyami SA, Moni MA (2021) Improved transfer-learning-based facial recognition framework to detect autistic children at an early stage. Brain Sci 11(6):734
Alam MS, Rashid MM, Roy R, Faizabadi AR, Gupta KD, Ahsan MM (2022) Empirical study of autism spectrum disorder diagnosis using facial images by improved transfer learning approach. Bioengineering 9:710. https://doi.org/10.3390/bioengineering9110710
Khosla Y, Ramachandra P and Chaitra N (2021) Detection of Autistic Individuals Using Facial Images and Deep Learning. In: 2021 IEEE International Conference on Computation System and Information Technology for Sustainable Solutions (CSITSS), Bangalore, India, pp 1–5
Tufail AB, Anwar N, Othman MTB, Ullah I, Khan RA, Ma Y-K, Adhikari D, Rehman AU, Shafiq M, Hamam H (2022). Early-Stage Alzheimer’s Disease Categorization Using PET Neuroimaging Modality and Convolutional Neural Networks in the 2D and 3D Domains. In: Sensors (Vol 22, Issue 12, p 4609). MDPI AG. https://doi.org/10.3390/s22124609
Kaur G, Sinha R, Tiwari PK, Yadav SK, Pandey P, Raj R, Vashisth A, Rakhra M (2021) Face mask recognition system using CNN model. Neurosci Inform 2:100035
Acknowledgements
The authors extend their appreciation to the Deanship of Scientific Research at King Khalid University for funding this work through large group Research Project under Grant number RGP2/249/44.
Funding
Not applicable.
Author information
Authors and Affiliations
Contributions
A.K., K.J., and S.A. wrote the main manuscript text and provided conceptualization, methodology, and software. S.R., M.A.K., and A.A. prepared figures and provided data curation. M.M. and M.S. provided validation and investigation and edited the manuscript. All authors reviewed the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kanwal, A., Javed, K., Ali, S. et al. A hybrid framework for detection of autism using ConvNeXt-T and embedding clusters. J Supercomput 80, 8156–8178 (2024). https://doi.org/10.1007/s11227-023-05761-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-023-05761-8