skip to main content
10.1145/3122986.3123006acmconferencesArticle/Chapter ViewAbstractPublication PagesautomotiveuiConference Proceedingsconference-collections
research-article

Learning-by-Doing: Using Near Infrared Spectroscopy to Detect Habituation and Adaptation in Automated Driving

Published: 24 September 2017 Publication History

Abstract

The advent of automated features in modern vehicles requires human factors researchers to find measures other than driving behavior to anticipate the response of drivers in various contexts. Functional near-infrared spectroscopy (fNIRS) is one research tool that allows us to quantify the driver's mental state. However, the underlying mechanisms of fNIRS technology can limit the possible contexts for its application. The pervasive question arises, whether the measurement device at hand is suitable for the research topic in question and is it capable of detecting the phenomenon under investigation? We provide a proof of concept study demonstrating that significant habituation is present when drivers operate new automated driving systems and that fNIRS technology is suitable to detect said driver habituation effects. The study presented here was conducted in a driving simulator and investigated the drivers' cortical activation in three different modes of automation: manual, partially autonomous, and fully autonomous modes.

References

[1]
Jans Aasman, Gijsbertus Mulder, and Lambertus JM Mulder. Operator effort and the measurement of heart-rate variability. Human Factors: The Journal of the Human Factors and Ergonomics Society, 29(2):161--170, 1987.
[2]
Pavlo Antonenko, Fred Paas, Roland Grabner, and Tamara van Gog. Using electroencephalography to measure cognitive load. Educational Psychology Review, 22(4):425--438, 2010.
[3]
Hasan Ayaz, Banu Onaral, Kurtulus Izzetoglu, Patricia A Shewokis, Ryan McKendrick, and Raja Parasuraman. Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: empirical examples and a technological development. Front. Hum. Neurosci, 7, 2013.
[4]
Nicholas A Badcock, Petroula Mousikou, Yatin Mahajan, Peter de Lissa, Johnson Thie, and Genevieve McArthur. Validation of the emotiv epoc® eeg gaming system for measuring research quality auditory erps. Peer J, 1:e38, 2013.
[5]
Joseph M Baker, Ning Liu, Xu Cui, Pascal Vrticka, Manish Saggar, SM Hadi Hosseini, and Allan L Reiss. Sex differences in neural and behavioral signatures of cooperation revealed by fnirs hyperscanning. Scientific reports, 6, 2016.
[6]
Aron K Barbey, Michael Koenigs, and Jordan Grafman. Dorsolateral prefrontal contributions to human working memory. Cortex, 49(5):1195--1205, 2013.
[7]
Mark E Bouton. Learning and behavior: A contemporary synthesis. Sinauer Associates, 2007.
[8]
Scott C Bunce, Kurtulus Izzetoglu, Hasan Ayaz, Patricia Shewokis, Meltem Izzetoglu, Kambiz Pourrezaei, and Banu Onaral. Implementation of fnirs for monitoring levels of expertise and mental workload. In International Conference on Foundations of Augmented Cognition, pages 13--22. Springer, 2011.
[9]
Christian Collet, Claire Petit, Stephane Champely, and Andre Dittmar. Assessing workload through physiological measurements in bus drivers using an automated system during docking. Human Factors: The Journal of the Human Factors and Ergonomics Society, 45(4):539--548, 2003.
[10]
BMW Electrek. BMW will launch the electric and autonomous iNext in 2021, new i8 in 2018 and not much in-between. https://electrek.co/2016/05/12/bmw-electric-autonomous-inext-2021/. Created: 2016-05-12.
[11]
Frank Fishburn, Megan Norr, Andrei Medvedev, and Chandan Vaidya. Sensitivity of fnirs to cognitive state and load. Frontiers in Human Neuroscience, 8:76, 2014.
[12]
Ford. Ford plans self-driving car for ride share fleets in 2021. Fordplansself-drivingcarforridesharefleetsin2021. Created: 2016-09-16.
[13]
Audrey Girouard, Erin Treacy Solovey, Leanne M Hirshfield, Evan M Peck, Krysta Chauncey, Angelo Sassaroli, Sergio Fantini, and Robert JK Jacob. From brain signals to adaptive interfaces: using fnirs in hci. In Brain-Computer Interfaces, pages 221--237. Springer, 2010.
[14]
Leanne M Hirshfield, Rebecca Gulotta, Stuart Hirshfield, Sam Hincks, Matthew Russell, Rachel Ward, Tom Williams, and Robert Jacob. This is your brain on interfaces: enhancing usability testing with functional near-infrared spectroscopy. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 373--382. ACM, 2011.
[15]
Theodore J. Huppert, Solomon G. Diamond, Maria A. Franceschini, and David A. Boas. Homer: a review of time-series analysis methods for near-infrared spectroscopy of the brain. Appl. Opt., 48(10):D280--D298, Apr 2009.
[16]
Takahiro Imai, Takanori Sato, Isao Nambu, and Yasuhiro Wada. Estimating brain activity of motor learning by using fnirs-glm analysis. In Neural Information Processing, pages 401--408. Springer, 2012.
[17]
Cécile Issard and Judit Gervain. Adult-like processing of time-compressed speech by newborns: A nirs study. Developmental cognitive neuroscience, 25:176--184, 2017.
[18]
Tzyy-Ping Jung, Scott Makeig, Magnus Stensmo, and Terrence J Sejnowski. Estimating alertness from the eeg power spectrum. Biomedical Engineering, IEEE Transactions on, 44(1):60--69, 1997.
[19]
Michael J Kane and Randall W Engle. The role of prefrontal cortex in working-memory capacity, executive attention, and general fluid intelligence: An individual-differences perspective. Psychonomic bulletin & review, 9(4):637--671, 2002.
[20]
Yi-Ou Li, Tom Eichele, Vince D Calhoun, and Tulay Adali. Group study of simulated driving fmri data by multiset canonical correlation analysis. Journal of signal processing systems, 68(1):31--48, 2012.
[21]
Ryan Mckendrick, Raja Parasuraman, Rabia Murtza, Alice Formwalt, Wendy Baccus, Martin Paczynski, and Hasan Ayaz. Into the wild: Neuroergonomic differentiation of hand-held and augmented reality wearable displays during outdoor navigation with functional near infrared spectroscopy. Frontiers in Human Neuroscience, 10:216, 2016.
[22]
David Miller, Annabel Sun, Mishel Johns, Hillary Ive, David Sirkin, Sudipto Aich, and Wendy Ju. Distraction becomes engagement in automated driving. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 59, pages 1676--1680. SAGE Publications Sage CA: Los Angeles, CA, 2015.
[23]
Earl K Miller and Jonathan D Cohen. An integrative theory of prefrontal cortex function. Annual review of neuroscience, 24(1):167--202, 2001.
[24]
Brian Mok, Mishel Johns, David Miller, and Wendy Ju. Tunneled in: Drivers with active secondary tasks need more time to transition from automation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pages 2840--2844. ACM, 2017.
[25]
Narender Ramnani and Adrian M Owen. Anterior prefrontal cortex: Insights into function from anatomy and neuroimaging. Nature Reviews Neuroscience, 5(3):184--194, 2004.
[26]
Holly EB Russell, Lene K Harbott, Ilana Nisky, Selina Pan, Allison M Okamura, and J Christian Gerdes. Motor learning affects car-to-driver handover in automated vehicles. Science Robotics, 1(1):eaah5682, 2016.
[27]
Taxonomy SAE. Definitions for terms related to on-road motor vehicle automated driving systems. Technical report, Technical report, SAE International, 2014.
[28]
Shota Saito, Takahiro Imai, Takanori Sato, Isao Nambu, and Yasuhiro Wada. Evaluation of motor learning by fnirs signal analysis using a general linear model. In Neural Engineering (NER), 2013 6th International IEEE/EMBS Conference on, pages 529--532. IEEE, 2013.
[29]
Cornelia Setz, Bert Arnrich, Johannes Schumm, Roberto La Marca, G Troster, and Ulrike Ehlert. Discriminating stress from cognitive load using a wearable eda device. Information Technology in Biomedicine, IEEE Transactions on, 14(2):410--417, 2010.
[30]
Toshiyuki Shimizu, Satoru Hirose, Hideo Obara, Kazuki Yanagisawa, Hitoshi Tsunashima, Yoshitaka Marumo, Tomoki Haji, and Masato Taira. Measurement of frontal cortex brain activity attributable to the driving workload and increased attention. SAE International Journal of Passenger Cars-Mechanical Systems, 2(1):736--744, 2009.
[31]
Srinath Sibi, Hasan Ayaz, David P Kuhns, David M Sirkin, and Wendy Ju. Monitoring driver cognitive load using functional near infrared spectroscopy in partially autonomous cars. In Intelligent Vehicles Symposium (IV), 2016 IEEE, pages 419--425. IEEE, 2016.
[32]
Erin T Solovey, Marin Zec, Enrique Abdon Garcia Perez, Bryan Reimer, and Bruce Mehler. Classifying driver workload using physiological and driving performance data: two field studies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 4057--4066. ACM, 2014.
[33]
Erin Treacy Solovey, Audrey Girouard, Krysta Chauncey, Leanne M Hirshfield, Angelo Sassaroli, Feng Zheng, Sergio Fantini, and Robert JK Jacob. Using fnirs brain sensing in realistic hci settings: experiments and guidelines. In Proceedings of the 22nd annual ACM symposium on User interface software and technology, pages 157--166. ACM, 2009.
[34]
Jun Tanaka, S Ishida, H Kawagoe, and S Kondo. Workload of using a driver assistance system. In Intelligent Transportation Systems, 2000. Proceedings. 2000 IEEE, pages 382--386. IEEE, 2000.
[35]
Tesla. Tesla Introduces Autopilot. http://www.cnbc.com/2015/10/14/tesla-rolls-out-autopilot-technology.html. Created: 2015-10-14.
[36]
Hitoshi Tsunashima and Kazuki Yanagisawa. Measurement of brain function of car driver using functional near-infrared spectroscopy (fnirs). Computational intelligence and neuroscience, 2009, 2009.
[37]
Shuhei Yamaguchi, Laura A Hale, Mark D'esposito, and Robert T Knight. Rapid prefrontal-hippocampal habituation to novel events. Journal of Neuroscience, 24(23):5356--5363, 2004.
[38]
Kayoko Yoshino, Noriyuki Oka, Kouji Yamamoto, Hideki Takahashi, and Toshinori Kato. Functional brain imaging using near-infrared spectroscopy during actual driving on an expressway. Frontiers in human neuroscience, 7:882, 2013.

Cited By

View all
  • (2024)An fNIRS dataset for driving risk cognition of passengers in highly automated driving scenariosScientific Data10.1038/s41597-024-03353-611:1Online publication date: 28-May-2024
  • (2024)Prefrontal Correlates of Passengers’ Mental Activity Based on fNIRS for High-Level Automated VehiclesAutomotive Innovation10.1007/s42154-023-00252-17:3(383-389)Online publication date: 13-May-2024
  • (2022)Driver Emotions in Automated VehiclesUser Experience Design in the Era of Automated Driving10.1007/978-3-030-77726-5_4(85-97)Online publication date: 1-Jan-2022
  • Show More Cited By

Index Terms

  1. Learning-by-Doing: Using Near Infrared Spectroscopy to Detect Habituation and Adaptation in Automated Driving

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    AutomotiveUI '17: Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications
    September 2017
    317 pages
    ISBN:9781450351508
    DOI:10.1145/3122986
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 24 September 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Autonomous Vehicles
    2. Cortical activation
    3. Driver habituation
    4. Human Factors
    5. fNIRS

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    AutomotiveUI '17
    Sponsor:

    Acceptance Rates

    AutomotiveUI '17 Paper Acceptance Rate 29 of 85 submissions, 34%;
    Overall Acceptance Rate 248 of 566 submissions, 44%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)24
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 21 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)An fNIRS dataset for driving risk cognition of passengers in highly automated driving scenariosScientific Data10.1038/s41597-024-03353-611:1Online publication date: 28-May-2024
    • (2024)Prefrontal Correlates of Passengers’ Mental Activity Based on fNIRS for High-Level Automated VehiclesAutomotive Innovation10.1007/s42154-023-00252-17:3(383-389)Online publication date: 13-May-2024
    • (2022)Driver Emotions in Automated VehiclesUser Experience Design in the Era of Automated Driving10.1007/978-3-030-77726-5_4(85-97)Online publication date: 1-Jan-2022
    • (2021)One-Dimensional Convolutional Neural Network Model for Abnormal Driving Behaviors Detection Using Smartphone Sensors2021 International Conference on Networking Systems of AI (INSAI)10.1109/INSAI54028.2021.00035(143-150)Online publication date: Nov-2021
    • (2021)Shedding light on the prefrontal correlates of mental workload in simulated driving: a functional near-infrared spectroscopy studyScientific Reports10.1038/s41598-020-80477-w11:1Online publication date: 12-Jan-2021
    • (2020)Where We Come from and Where We Are Going: A Systematic Review of Human Factors Research in Driving AutomationApplied Sciences10.3390/app1024891410:24(8914)Online publication date: 14-Dec-2020
    • (2019)Methodological Approaches and Recommendations for Functional Near-Infrared Spectroscopy Applications in HF/E ResearchHuman Factors: The Journal of the Human Factors and Ergonomics Society10.1177/001872081984527562:4(613-642)Online publication date: 20-May-2019
    • (2019)Workload Measures—Recent Trends in the Driving ContextWandel durch Partizipation10.1007/978-3-030-14730-3_45(419-430)Online publication date: 28-Feb-2019

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media