skip to main content
10.1145/3411763.3451784acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
poster

EyeXplain Autism: Interactive System for Eye Tracking Data Analysis and Deep Neural Network Interpretation for Autism Spectrum Disorder Diagnosis

Published: 08 May 2021 Publication History

Abstract

Over the past decade, Deep Neural Networks (DNN) applied to eye tracking data have seen tremendous progress in their ability to perform Autism Spectrum Disorder (ASD) diagnosis. Despite their promising accuracy, DNNs are often seen as ’black boxes’ by physicians unfamiliar with the technology. In this paper, we present EyeXplain Autism, an interactive system that enables physicians to analyse eye tracking data, perform automated diagnosis and interpret DNN predictions. Here we discuss the design, development and sample scenario to illustrate the potential of our system to aid in ASD diagnosis. Unlike existing eye tracking software, our system combines traditional eye tracking visualisation and analysis tools with a data-driven knowledge to enhance medical decision-making for physicians.

References

[1]
Karan Ahuja, Abhishek Bose, Mohit Jain, Kuntal Dey, Anil Joshi, Krishnaveni Achary, Blessin Varkey, Chris Harrison, and Mayank Goel. 2020. Gaze-based Screening of Autistic Traits for Adolescents and Young Adults using Prosaic Videos. In Proceedings of the 3rd ACM SIGCAS Conference on Computing and Sustainable Societies. 324–324.
[2]
Tanja Blascheck, Kuno Kurzhals, Michael Raschke, Michael Burch, Daniel Weiskopf, and Thomas Ertl. 2014. State-of-the-Art of Visualization for Eye Tracking Data. In EuroVis (STARs).
[3]
Tanja Blascheck, Kuno Kurzhals, Michael Raschke, Michael Burch, Daniel Weiskopf, and Thomas Ertl. 2017. Visualization of eye tracking data: A taxonomy and survey. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 260–284.
[4]
Ali Borji. 2019. Saliency prediction in the deep learning era: Successes and limitations. IEEE transactions on pattern analysis and machine intelligence (2019).
[5]
Shi Chen and Qi Zhao. 2019. Attention-based autism spectrum disorder screening with privileged modality. In Proceedings of the IEEE International Conference on Computer Vision. 1181–1190.
[6]
Ryan Anthony J de Belen, Tomasz Bednarz, Arcot Sowmya, and Dennis Del Favero. 2020. Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019. Translational psychiatry 10, 1 (2020), 1–20.
[7]
Huiyu Duan, Xiongkuo Min, Yi Fang, Lei Fan, Xiaokang Yang, and Guangtao Zhai. 2019. Visual Attention Analysis and Prediction on Human Faces for Children with Autism Spectrum Disorder. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 15, 3s (2019), 1–23.
[8]
Huiyu Duan, Guangtao Zhai, Xiongkuo Min, Yi Fang, Zhaohui Che, Xiaokang Yang, Cheng Zhi, Hua Yang, and Ning Liu. 2018. Learning to predict where the children with asd look. In 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, 704–708.
[9]
V Eapan, A Masi, and F Kahn. 2020. Australian Autism Biobank follow-up cohort pilot study: Final Report. Cooperative Research Centre for Living with Autism, Brisbane. Copies of this report can be downloaded from the Autism CRC website autismcrc. com. au (2020).
[10]
Annette Estes, Jeffrey Munson, Sally J Rogers, Jessica Greenson, Jamie Winter, and Geraldine Dawson. 2015. Long-term outcomes of early intervention in 6-year-old children with autism spectrum disorder. Journal of the American Academy of Child & Adolescent Psychiatry 54, 7(2015), 580–587.
[11]
Yi Fang, Huiyu Duan, Fangyu Shi, Xiongkuo Min, and Guangtao Zhai. 2020. Identifying Children with Autism Spectrum Disorder Based on Gaze-Following. In 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 423–427.
[12]
Yuming Fang, Hanqin Huang, Boyang Wan, and Yifan Zuo. 2019. Visual Attention Modeling for Autism Spectrum Disorder by Semantic Features. In 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE, 625–628.
[13]
Jesús Gutiérrez, Zhaohui Che, Guangtao Zhai, and Patrick Le Callet. 2021. Saliency4ASD: Challenge, dataset and tools for visual attention modeling for autism spectrum disorder. Signal Processing: Image Communication 92 (2021), 116092.
[14]
Xun Huang, Chengyao Shen, Xavier Boix, and Qi Zhao. 2015. Salicon: Reducing the semantic gap in saliency prediction by adapting deep neural networks. In Proceedings of the IEEE International Conference on Computer Vision. 262–270.
[15]
Marisela Huerta, Somer L Bishop, Amie Duncan, Vanessa Hus, and Catherine Lord. 2012. Application of DSM-5 criteria for autism spectrum disorder to three samples of children with DSM-IV diagnoses of pervasive developmental disorders. American Journal of Psychiatry 169, 10 (2012), 1056–1064.
[16]
Ming Jiang, Sunday M Francis, Diksha Srishyla, Christine Conelea, Qi Zhao, and Suma Jacob. 2019. Classifying individuals with ASD through facial emotion recognition and eye-tracking. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 6063–6068.
[17]
Ming Jiang and Qi Zhao. 2017. Learning visual attention to identify people with autism spectrum disorder. In Proceedings of the IEEE International Conference on Computer Vision. 3267–3276.
[18]
Ayush Kumar, Prantik Howlader, Rafael Garcia, Daniel Weiskopf, and Klaus Mueller. 2020. Challenges in Interpretability of Neural Networks for Eye Movement Data. In ETRA Short Papers. 12–1.
[19]
Olivier Le Meur, Alexis Nebout, Myriam Cherel, and Elise Etchamendy. 2020. From Kanner Austim to Asperger Syndromes, the Difficult Task to Predict Where ASD People Look at. IEEE Access 8(2020), 162132–162140.
[20]
Sidrah Liaqat, Chongruo Wu, Prashanth Reddy Duggirala, Sen-ching Samson Cheung, Chen-Nee Chuah, Sally Ozonoff, and Gregory Young. 2021. Predicting ASD diagnosis in children with synthetic and image-based eye gaze data. Signal Processing: Image Communication(2021), 116198.
[21]
Wenbo Liu, Ming Li, and Li Yi. 2016. Identifying children with autism spectrum disorder based on their face processing abnormality: A machine learning framework. Autism Research 9, 8 (2016), 888–898.
[22]
Catherine Lord, Michael Rutter, Susan Goode, Jacquelyn Heemsbergen, Heather Jordan, Lynn Mawhood, and Eric Schopler. 1989. Austism diagnostic observation schedule: A standardized observation of communicative and social behavior. Journal of autism and developmental disorders 19, 2 (1989), 185–212.
[23]
Catherine Lord, Michael Rutter, and Ann Le Couteur. 1994. Autism Diagnostic Interview-Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of autism and developmental disorders 24, 5 (1994), 659–685.
[24]
Ann M Mastergeorge, Chanaka Kahathuduwa, and Jessica Blume. 2020. Eye-tracking in infants and young children at risk for autism spectrum disorder: A systematic review of visual stimuli in experimental paradigms. Journal of Autism and Developmental Disorders (2020), 1–22.
[25]
Pramit Mazumdar, Giuliano Arru, and Federica Battisti. 2021. Early detection of children with autism spectrum disorder based on visual exploration of images. Signal Processing: Image Communication(2021), 116184.
[26]
Adrienne Moore, Madeline Wozniak, Andrew Yousef, Cindy Carter Barnes, Debra Cha, Eric Courchesne, and Karen Pierce. 2018. The geometric preference subtype in ASD: identifying a consistent, early-emerging phenomenon through eye tracking. Molecular autism 9, 1 (2018), 19.
[27]
Alexis Nebout, Weijie Wei, Zhi Liu, Lijin Huang, and Olivier Le Meur. 2019. Predicting Saliency Maps for ASD People. In 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE, 629–632.
[28]
Anneli Olsen. 2012. The Tobii I-VT fixation filter. Tobii Technology (2012), 1–21.
[29]
Karen Pierce, David Conant, Roxana Hazin, Richard Stoner, and Jamie Desmond. 2011. Preference for geometric patterns early in life as a risk factor for autism. Archives of general psychiatry 68, 1 (2011), 101–109.
[30]
Karen Pierce, Steven Marinero, Roxana Hazin, Benjamin McKenna, Cynthia Carter Barnes, and Ajith Malige. 2016. Eye tracking reveals abnormal visual preference for geometric images as an early biomarker of an autism spectrum disorder subtype associated with increased symptom severity. Biological psychiatry 79, 8 (2016), 657–666.
[31]
Yudong Tao and Mei-Ling Shyu. 2019. SP-ASDNet: CNN-LSTM based ASD classification model using observer scanpaths. In 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE, 641–646.
[32]
Vladimir N Vapnik. 1999. An overview of statistical learning theory. IEEE transactions on neural networks 10, 5 (1999), 988–999.
[33]
Wenguan Wang, Jianbing Shen, Fang Guo, Ming-Ming Cheng, and Ali Borji. 2018. Revisiting video saliency: A large-scale benchmark and a new model. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4894–4903.
[34]
Weijie Wei, Zhi Liu, Lijin Huang, Alexis Nebout, and Olivier Le Meur. 2019. Saliency prediction via multi-level features and deep supervision for children with autism spectrum disorder. In 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE, 621–624.
[35]
Weijie Wei, Zhi Liu, Lijin Huang, Alexis Nebout, Olivier Le Meur, Tianhong Zhang, Jijun Wang, and Lihua Xu. 2020. Predicting atypical visual saliency for autism spectrum disorder via scale-adaptive inception module and discriminative region enhancement loss. Neurocomputing (2020).
[36]
Weijie Wei, Zhi Liu, Lijin Huang, Ziqiang Wang, Weiyu Chen, Tianhong Zhang, Jijun Wang, and Lihua Xu. 2021. Identify autism spectrum disorder via dynamic filter and deep spatiotemporal feature extraction. Signal Processing: Image Communication(2021), 116195.
[37]
Yao Xie, Melody Chen, David Kao, Ge Gao, and Xiang’Anthony’ Chen. 2020. CheXplain: Enabling Physicians to Explore and Understand Data-Driven, AI-Enabled Medical Imaging Analysis. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.

Cited By

View all
  • (2024)Enhancing Autism Detection Through Gaze Analysis Using Eye Tracking Sensors and Data Attribution with Distillation in Deep Neural NetworksSensors10.3390/s2423779224:23(7792)Online publication date: 5-Dec-2024
  • (2024)TwIPS: A Large Language Model Powered Texting Application to Simplify Conversational Nuances for Autistic UsersProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675633(1-18)Online publication date: 27-Oct-2024
  • (2024)Early Autism Screening in Children Using Facial RecognitionExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651045(1-7)Online publication date: 11-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems
May 2021
2965 pages
ISBN:9781450380959
DOI:10.1145/3411763
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Autism Diagnosis
  2. Deep Neural Network
  3. Explainable AI
  4. Eye Tracking
  5. Visualization

Qualifiers

  • Poster
  • Research
  • Refereed limited

Conference

CHI '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)79
  • Downloads (Last 6 weeks)7
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Enhancing Autism Detection Through Gaze Analysis Using Eye Tracking Sensors and Data Attribution with Distillation in Deep Neural NetworksSensors10.3390/s2423779224:23(7792)Online publication date: 5-Dec-2024
  • (2024)TwIPS: A Large Language Model Powered Texting Application to Simplify Conversational Nuances for Autistic UsersProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675633(1-18)Online publication date: 27-Oct-2024
  • (2024)Early Autism Screening in Children Using Facial RecognitionExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651045(1-7)Online publication date: 11-May-2024
  • (2024)Deep learning with image-based autism spectrum disorder analysisEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107185127:PAOnline publication date: 1-Feb-2024
  • (2023)Eye-tracking correlates of response to joint attention in preschool children with autism spectrum disorderBMC Psychiatry10.1186/s12888-023-04585-323:1Online publication date: 29-Mar-2023
  • (2023)Machine Learning Model to Predict Autism Spectrum Disorder Using Eye Gaze Tracking2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM58861.2023.10386016(4002-4006)Online publication date: 5-Dec-2023
  • (2023)Eye Tracking Biomarkers for Autism Spectrum Disorder Detection using Machine Learning and Deep Learning Techniques: ReviewResearch in Autism Spectrum Disorders10.1016/j.rasd.2023.102228108(102228)Online publication date: Oct-2023
  • (2022)Eye Tracking-Based Diagnosis and Early Detection of Autism Spectrum Disorder Using Machine Learning and Deep Learning TechniquesElectronics10.3390/electronics1104053011:4(530)Online publication date: 10-Feb-2022
  • (2022)Real-time Gaze Tracking with Head-eye Coordination for Head-mounted Displays2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR55827.2022.00022(82-91)Online publication date: Oct-2022
  • (2022)What Are the Users’ Needs? Design of a User-Centered Explainable Artificial Intelligence Diagnostic SystemInternational Journal of Human–Computer Interaction10.1080/10447318.2022.209509339:7(1519-1542)Online publication date: 26-Jul-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media