research-article

Design of a Physiology-based Adaptive Virtual Reality Driving Platform for Individuals with ASD

Authors:

Zachary Warren,

Nilanjan SarkarAuthors Info & Claims

ACM Transactions on Accessible Computing (TACCESS), Volume 12, Issue 1

Article No.: 2, Pages 1 - 24

https://doi.org/10.1145/3301498

Published: 11 February 2019 Publication History

Abstract

Driving is essential for many people in developed countries to achieve independence. Individuals with Autism Spectrum Disorder (ASD), in addition to having social skill deficits, may experience difficulty in learning to drive due to deficits in attention-shifting, performing sequential tasks, integrating visual-motor responses, and coordinating motor response. Lacking confidence and feeling anxiety further exacerbates these concerns. While there is a growing body of research regarding assessment of driving behavior or comparisons of driving behaviors between individuals with and without ASD, there is a lack of driving simulator that is catered toward the needs of individuals with ASD. We present the development of a novel closed-loop adaptive Virtual Reality (VR) driving simulator for individuals with ASD that can infer one's engagement based on his/her physiological responses and adapts driving task difficulty based on engagement level in real-time. We believe that this simulator will provide opportunities for learning driving skills in a safe and individualized environment to individuals with ASD and help them with independent living. We also conducted a small user study with teenagers with ASD to demonstrate the feasibility and tolerability of such a driving simulator. Preliminary results showed that the participants found the engagement-sensitive system more engaging and more enjoyable than a purely performance-sensitive system. These findings could support future work into driving simulator technologies, which could provide opportunities to practice driving skills in cost-effective, supportive, and safe environments.

References

[1]

E. Adams. 2008. The designer's notebook: Difficulty modes and dynamic difficulty adjustment. Gamasutra. Retrieved from http://www.gamasutra.com/.

[2]

American Psychiatric Association. 2013. Diagnostic and Statistical Manual of Mental Disorders (DSM-5). American Psychiatric Pub.

[3]

J. Baio, L. Wiggins, D. L. Christensen, et al. 2018. Prevalence and characteristics of autism spectrum disorder among children aged 8 years—Autism and developmental disabilities monitoring network, 11 sites, United States, 2014. MMWR. Surveill. Sum. 67 (2018), 1--23.

[4]

B. B. Bederson. 2004. Interfaces for staying in the flow. Ubiquity (2004), 1.

Digital Library

[5]

M. Bellani, L. Fornasari, L. Chittaro, and P. Brambilla. 2011. Virtual reality in autism: State of the art. Epidemiol. Psych. Sci. 20, 3 (2011), 235--238.

[6]

E. Bekele, et al. 2016. Multimodal adaptive social interaction in virtual environment (MASI-VR) for children with Autism spectrum disorders (ASD). In IEEE Virtual Reality (VR), Greenville, SC. 121--130.

[7]

D. Bian, J. Wade, A. Swanson, Z. Warren, and N. Sarkar. 2015. Physiology-based affect recognition during driving in virtual environment for autism intervention. In Proceedings of the 2nd International Conference on Physiological Computing Systems.

Digital Library

[8]

D. Bian, J. Wade, Z. Warren, and N. Sarkar. 2016. Online engagement detection and task adaptation in a virtual reality based driving simulator for autism intervention. Proceedings of the International Conference on Universal Access in Human-Computer Interaction. Springer.

[9]

D. Bian, J. W. Wade, L. Zhang, E. Bekele, A. Swanson, J. A. Crittendon, M. Sarkar, Z. Warren, and N. Sarkar. 2013. A novel virtual reality driving environment for autism intervention. In Proceedings of the Universal Access in Human-Computer Interaction. User and Context Diversity. Springer, 474--483.

Digital Library

[10]

A. V. Buescher, Z. Cidav, M. Knapp, and D. S. Mandell. 2014. Costs of autism spectrum disorders in the United Kingdom and the United States. JAMA Pediatr. 168, 8 (2014), 721--728.

[11]

Centers for Disease Control and Prevention. 2012. Injury Prevention and Control: Motor Vehicle Safety—Distracted Driving. https://www.cdc.gov/motorvehiclesafety/distracted_driving/index.html.

[12]

J. Baio, L. Wiggins, D. L. Christensen, et al. Prevalence of autism spectrum disorder among children aged 8 years - Autism and developmental disabilities monitoring network, 11 Sites, United States, 2014. 2018. MMWR Surveill Summ. 67, 6 (2018), 1--23.

[13]

G. Chanel, C. Rebetez, M. Bétrancourt, and T. Pun. 2011. Emotion assessment from physiological signals for adaptation of game difficulty. IEEE Trans. Syst. Man Cybernet. Part A: Syst. Hum. 41, 6 (2011), 1052--1063.

Digital Library

[14]

G. S. Chasson, G. E. Harris, and W. J. Neely. 2007. Cost comparison of early intensive behavioral intervention and special education for children with autism. J. Child Fam. Studies 16, 3 (2007), 401--413.

[15]

S. Classen, M. Monahan, and Y. Wang. 2013. Driving characteristics of teens with attention deficit hyperactivity and autism spectrum disorder. Amer. J. Occupat. Ther. 67, 6 (2013), 664--673.

[16]

J. N. Constantino and C. P. Gruber. 2007. Social Responsiveness Scale (SRS). Western Psychological Services, Los Angeles, CA.

[17]

N. Cox, R. Reeve, S. Cox, and D. Cox. 2012. Brief report: Driving and young adults with ASD: Parents’ experiences. J. Autism Dev. Disorders 42, 10 (2012), 2257--2262.

[18]

S. M. Cox, D. J. Cox, M. J. Kofler, M. A. Moncrief, R. J. Johnson, A. E. Lambert, S. A. Cain, and R. E. Reeve. 2016. Driving simulator performance in novice drivers with autism spectrum disorder: The role of executive functions and basic motor skills. J. Autism Dev. Disorders 46, 4 (2016), 1379--1391.

[19]

B. P. Daly, E. G. Nicholls, K. E. Patrick, D. D. Brinckman, and M. T. Schultheis. 2014. Driving behaviors in adults with autism spectrum disorders. J. Autism Dev. Disorders 44, 12 (2014), 3119--3128.

[20]

S. H. Fairclough, K. Gilleade, K. C. Ewing, and J. Roberts. 2013. Capturing user engagement via psychophysiology: Measures and mechanisms for biocybernetic adaptation. Int. J. Auton. Adapt. Commun. Syst. 6, 1 (2013), 63--79.

Digital Library

[21]

M. L. Ganz. 2006. The Costs of Autism. CRC Press, New York, 475--502.

[22]

D. Giakoumis, A. Vogiannou, I. Kosunen, K. Moustakas, D. Tzovaras, and G. Hassapis. 2010. Identifying Psychophysiological Correlates of Boredom and Negative Mood Induced during HCI. B-Interface.

[23]

M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. 2009. The WEKA data mining software: An update. ACM SIGKDD Explor. Newslett. 11, 1 (2009), 10--18.

Digital Library

[24]

E. L. Hill. 2004. Evaluating the theory of executive dysfunction in autism. Develop. Rev. 24, 2 (2004), 189--233.

[25]

L. M. Howden and J. A. Meyer. 2010. Age and sex composition: 2010. 2010 Census Briefs, U.S. Department of Commerce, Economics and Statistics Administration. U.S. Census Bureau.

[26]

P. Huang, T. Kao, A. E. Curry, and D. R. Durbin. 2012. Factors associated with driving in teens with autism spectrum disorders. J. Dev. Behav. Pediatr. 33, 1 (2012), 70--74.

[27]

R. Hunicke. 2005. The case for dynamic difficulty adjustment in games. In Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology. ACM.

Digital Library

[28]

K. Jegers. 2007. Pervasive game flow: Understanding player enjoyment in pervasive gaming. Comput. Entertain. 5, 1 (2007), 9.

Digital Library

[29]

S. Jerritta, M. Murugappan, R. Nagarajan, and K. Wan. 2011. Physiological signals-based human emotion recognition: A review. In Proceedings of the IEEE 7th International Colloquium on Signal Processing and its Applications (CSPA’11). IEEE.

[30]

J. Kim and E. André. 2008. Emotion recognition based on physiological changes in music listening. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 12 (2008), 2067--2083.

Digital Library

[31]

U. Lahiri, E. Bekele, E. Dohrmann, Z. Warren, and N. Sarkar. 2013. Design of a virtual reality-based adaptive response technology for children with autism. IEEE Trans. Neural Syst. Rehab. Eng. 21, 1 (2013), 55--64.

[32]

C. Liu, K. Conn, N. Sarkar, and W. Stone. 2008. Online affect detection and robot behavior adaptation for intervention of children with autism. IEEE Trans. Robot. 24, 4 (2008), 883--896.

Digital Library

[33]

C. Lord, S. Risi, L. Lambrecht, E. H. Cook, B. L. Leventhal, P. C. DiLavore, A. Pickles, and M. Rutter. 2000. The autism diagnostic observation schedule—generic: A standard measure of social and communication deficits associated with the spectrum of autism. J. Autism Dev. Disorders 30, 3 (2000), 205--223.

[34]

L. Nacke and C. A. Lindley. 2008. Flow and immersion in first-person shooters: Measuring the player's gameplay experience. In Proceedings of the 2008 Conference on Future Play: Research, Play, Share. ACM.

Digital Library

[35]

National Highway Traffic Safety Administration. 2016. 2015 motor vehicle crashes: Overview (DOT HS 812 456). Traffic Safety Facts Research Note (2016), 1--9. https://crashstats.nhtsa.dot.gov/Api/Public/Publication/812456.

[36]

National Highway Traffic Safety Administration. 2018. 2016 motor vehicle crashes: Overview (DOT HS 812 498). Traffic Safety Facts Research Note (2018). https://crashstats.nhtsa.dot.gov/Api/Public/Publication/812498.

[37]

K. A. Orvis, D. B. Horn, and J. Belanich. 2008. The roles of task difficulty and prior videogame experience on performance and motivation in instructional videogames. Comput. Hum. Behav. 24, 5 (2008), 2415--2433.

Digital Library

[38]

A. Pecchinenda. 1996. The affective significance of skin conductance activity during a difficult problem-solving task. Cogn. Emot. 10, 5 (1996), 481--504.

[39]

R. Pekrun, T. Goetz, L. M. Daniels, R. H. Stupnisky, and R. P. Perry. 2010. Boredom in achievement settings: Exploring control--value antecedents and performance outcomes of a neglected emotion. J. Edu. Psychol. 102, 3 (2010), 531.

[40]

J. A. Healey. 2000. Wearable and automotive systems for affect recognition from physiology. Massachusetts Institute of Technology. http://hdl.handle.net/1721.1/9067.

[41]

P. Rainville, A. Bechara, N. Naqvi, and A. R. Damasio. 2006. Basic emotions are associated with distinct patterns of cardiorespiratory activity. Int. J. Psychophysiol. 61, 1 (2006), 5--18.

[42]

P. Rani, N. Sarkar, and C. Liu. 2005. Maintaining optimal challenge in computer games through real-time physiological feedback. In Proceedings of the 11th International Conference on Human Computer Interaction.

[43]

B. Reimer, R. Fried, B. Mehler, G. Joshi, A. Bolfek, K. Godfrey, N. Zhao, R. Goldin, and J. Biederman. 2013. Brief report: Examining driving behavior in young adults with high functioning autism spectrum disorders: A pilot study using a driving simulation paradigm. J. Autism Dev. Disorders. 1--7.

[44]

M. Rutter, A. Bailey, and C. Lord. 2003. The Social Communication Questionnaire: Manual. Western Psychological Services.

[45]

N. Sarkar. 2002. Psychophysiological control architecture for human-robot coordination-concepts and initial experiments. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’02). IEEE.

[46]

O. Shechtman, S. Classen, K. Awadzi, and W. Mann. 2009. Comparison of driving errors between on-the-road and simulated driving assessment: A validation study. Traffic Injury Prevent. 10, 4 (2009), 379--385.

[47]

E. Sheppard, E. van Loon, G. Underwood, and D. Ropar. 2016. Difficulties predicting time-to-arrival in individuals with autism spectrum disorders. Res. Autism Spectrum Disorders 28 (2016), 17--23.

[48]

G. Stemmler, M. Heldmann, C. A. Pauls, and T. Scherer. 2001. Constraints for emotion specificity in fear and anger: The context counts. Psychophysiology 38, 2 (2001), 275--291.

[49]

D. Strickland. 1997. Virtual reality for the treatment of autism. Studies Health Technol. Info. 44 (1997), 81--86.

[50]

R. Stripling, J. T. Coyne, A. Cole, D. Afergan, R. L. Barnes, K. A. Rossi, L. M. Reeves, and D. D. Schmorrow. 2007. Automated SAF adaptation tool (ASAT). In Proceedings of the International Conference on Foundations of Augmented Cognition. Springer.

Digital Library

[51]

J. Wade, D. Bian, J. Fan, L. Zhang, A. Swanson, M. Sarkar, A. Weitlauf, Z. Warren, and N. Sarkar. 2015. A virtual reality driving environment for training safe gaze patterns: Application in individuals with ASD. In Proceedings of the Universal Access in Human-Computer Interaction. Access to Learning, Health and Well-Being. Springer, 689--697.

[52]

J. Wade, D. Bian, L. Zhang, A. Swanson, M. Sarkar, Z. Warren, and N. Sarkar. 2014. Design of a virtual reality driving environment to assess performance of teenagers with ASD. In Proceedings of the International Conference on Universal Access in Human-Computer Interaction. Springer International Publishing.

[53]

S. W. White, K. Keonig, and L. Scahill. 2007. Social skills development in children with autism spectrum disorders: A review of the intervention research. J. Autism Dev. Disorders 37, 10 (2007), 1858--1868.

[54]

Z. E. Warren, Z. Zheng, A. R. Swanson, E. Bekele, L. Zhang, J. A. Crittendon, A. F. Weitlauf, and N. Sarkar. 2015. Can robotic interaction improve joint attention skills? Journal of Autism and Developmental Disorders 45, 11 (2015), 3726--3734.

[55]

L. Zhang, Z. Warren, A. Swanson, et al. 2018. J Autism Dev. Disord. 48 (2018), 2779.

[56]

Z. Zheng, et al. 2015. Design of a computer-assisted system for teaching attentional skills to toddlers with ASD. In Universal Access in Human-Computer Interaction. Access to Learning, Health and Well-Being (UAHCI’15), M. Antona and C. Stephanidis (Eds.). Lecture Notes in Computer Science, Vol. 9177. Springer, Cham.

[57]

J. Zhou, J. Sun, F. Chen, Y. Wang, R. Taib, A. Khawaji, and Z. Li. 2015. Measurable decision making with GSR and pupillary analysis for intelligent user interface. ACM Trans. Comput.-Hum. Interact. 21, 6 (2015), 1--23.

Digital Library

Cited By

Adams JHamell JWalker P(2025)Can a Single Human Supervise a Swarm of 100 Heterogeneous Robots?IEEE Transactions on Field Robotics10.1109/TFR.2024.35023162(46-80)Online publication date: 2025
https://doi.org/10.1109/TFR.2024.3502316
Liu LYao XChen JZhang KLiu LWang GLing Y(2024)Virtual Reality Utilized for Safety Skills Training for Autistic Individuals: A ReviewBehavioral Sciences10.3390/bs1402008214:2(82)Online publication date: 23-Jan-2024
https://doi.org/10.3390/bs14020082
Khan NPlunk AZheng ZAdiani DStaubitz JWeitlauf ASarkar N(2024)Pilot study of a real-time early agitation capture technology (REACT) for children with intellectual and developmental disabilitiesDIGITAL HEALTH10.1177/2055207624128788410Online publication date: 14-Oct-2024
https://doi.org/10.1177/20552076241287884
Show More Cited By

Index Terms

Design of a Physiology-based Adaptive Virtual Reality Driving Platform for Individuals with ASD
1. Computing methodologies
  1. Machine learning
2. Human-centered computing
  1. Accessibility
    1. Accessibility design and evaluation methods
  2. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. Usability testing

Recommendations

A Gaze-Contingent Adaptive Virtual Reality Driving Environment for Intervention in Individuals with Autism Spectrum Disorders
Special Issue on New Directions in Eye Gaze for Interactive Intelligent Systems (Part 2 of 2), Regular Articles and Special Issue on Highlights of IUI 2015 (Part 1 of 2)

In addition to social and behavioral deficits, individuals with Autism Spectrum Disorder (ASD) often struggle to develop the adaptive skills necessary to achieve independence. Driving intervention in individuals with ASD is a growing area of study, but ...
Sexting experiences and motivations among adolescents with ADHD and ASD
Abstract
Sexting (i.e., to send and receive self-produced sexual materials) has become a common element in adolescents’ socio-sexual exploration. Although often harmless, sexting also involves the risk of abuse, and potentially more so for some ...
Highlights
- Adolescents, including those with ADHD/ASD, mainly use sexting for sexual gratification and exploration.
Physiology-based Affect Recognition During Driving in Virtual Environment for Autism Intervention
PhyCS 2015: Proceedings of the 2nd International Conference on Physiological Computing Systems

Independent driving is believed to be an important factor of quality of life for individual with autism

spectrum disorder (ASD). In recent years, several computer technologies, particularly Virtual Reality (VR),

have been explored to improve driving ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Accessible Computing

ACM Transactions on Accessible Computing Volume 12, Issue 1

March 2019

90 pages

ISSN:1936-7228

EISSN:1936-7236

DOI:10.1145/3312747

Editors:
Matt Huenerfauth
Rochester Institute of Technology, USA
,
Kathleen F. McCoy
University of Delaware, USA

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 February 2019

Accepted: 01 November 2018

Revised: 01 September 2018

Received: 01 December 2017

Published in TACCESS Volume 12, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Institutes of Health

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

39
Total Citations
View Citations
1,001
Total Downloads

Downloads (Last 12 months)114
Downloads (Last 6 weeks)16

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Adams JHamell JWalker P(2025)Can a Single Human Supervise a Swarm of 100 Heterogeneous Robots?IEEE Transactions on Field Robotics10.1109/TFR.2024.35023162(46-80)Online publication date: 2025
https://doi.org/10.1109/TFR.2024.3502316
Liu LYao XChen JZhang KLiu LWang GLing Y(2024)Virtual Reality Utilized for Safety Skills Training for Autistic Individuals: A ReviewBehavioral Sciences10.3390/bs1402008214:2(82)Online publication date: 23-Jan-2024
https://doi.org/10.3390/bs14020082
Khan NPlunk AZheng ZAdiani DStaubitz JWeitlauf ASarkar N(2024)Pilot study of a real-time early agitation capture technology (REACT) for children with intellectual and developmental disabilitiesDIGITAL HEALTH10.1177/2055207624128788410Online publication date: 14-Oct-2024
https://doi.org/10.1177/20552076241287884
Finseth TDorneich MKeren NFranke WVardeman S(2024)Virtual Reality Adaptive Training for Personalized Stress InoculationHuman Factors: The Journal of the Human Factors and Ergonomics Society10.1177/0018720824124196867:1(5-20)Online publication date: 28-Mar-2024
https://doi.org/10.1177/00187208241241968
Hadadi AChardonnet JGuillet COvtcharova J(2024)Machine Learning Application for Real-Time SimulatorProceedings of the 2024 9th International Conference on Machine Learning Technologies10.1145/3674029.3674030(1-5)Online publication date: 24-May-2024
https://dl.acm.org/doi/10.1145/3674029.3674030
Migovich MAdiani DBreen MSwanson AVogus TSarkar N(2024)Stress Detection of Autistic Adults during Simulated Job Interviews Using a Novel Physiological Dataset and Machine LearningACM Transactions on Accessible Computing10.1145/363970917:1(1-25)Online publication date: 8-Jan-2024
https://dl.acm.org/doi/10.1145/3639709
Hadadi AChardonnet JGuillet COvtcharova J(2024)SmartSimVR: An Architecture Integrating Machine Learning and Virtual Environment for Real-Time Simulation Adaptation2024 10th International Conference on Automation, Robotics and Applications (ICARA)10.1109/ICARA60736.2024.10553156(531-535)Online publication date: 22-Feb-2024
https://doi.org/10.1109/ICARA60736.2024.10553156
Rahmani RWestin TNevelsteen K(2024)Future Healthcare in Generative AI with Real MetaverseProcedia Computer Science10.1016/j.procs.2024.11.137251(487-493)Online publication date: 2024
https://doi.org/10.1016/j.procs.2024.11.137
Hossain MSowers AClapp JNovak V(2024)Different adaptation error types in affective computing have different effects on user experience: a Wizard-of-Oz studyInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103440(103440)Online publication date: Dec-2024
https://doi.org/10.1016/j.ijhcs.2024.103440
Novak VHass DHossain MSowers AClapp J(2024)Effects of adaptation accuracy and magnitude in affect-aware difficulty adaptation for the multi-attribute task batteryInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103180183(103180)Online publication date: Mar-2024
https://doi.org/10.1016/j.ijhcs.2023.103180
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents