Skip to main content
Springer Nature Link
Log in

Effects of a Robot Human-Machine Interface on Emergency Steering Control and Prefrontal Cortex Activation in Automatic Driving

  • Conference paper
  • First Online:
Engineering Psychology and Cognitive Ergonomics (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13307))

Included in the following conference series:

  • 1850 Accesses

Abstract

Advanced driver assistance systems (ADASs) support drivers in multiple ways, such as adaptive cruise control, lane tracking assistance (LTA), and blind spot monitoring, among other services. However, the use of ADAS cruise control has been reported to delay reaction to vehicle collisions. We created a robot human-machine interface (RHMI) to inform drivers of emergencies by means of movement, which would allow drivers to prepare for the disconnection of autonomous driving. This study investigated the effects of RHMI on response to the emergency disconnection of the LTA function of autonomous driving. We also examined drivers’ fatigue and arousal using near-infrared spectroscopy (NIRS) on the prefrontal cortex. The participants in this study were 12 males and 15 females. We recorded steering torque and NIRS data in the prefrontal region across two channels during the manipulation of automatic driving with a driving simulator. The scenario included three events in the absence of LTA due to bad weather. All of the participants experienced emergencies with and without RHMI, implemented using two agents: RHMI prototype (RHMI-P) and RoBoHoN. Our RHMI allowed the drivers to respond earlier to emergency LTA disconnection. All drivers showed a gentle torque response for RoBoHoN, but some showed a steep response with RHMI-P and without RHMI. NIRS data showed significant prefrontal cortex activation in RHMI conditions (especially RHMI-P), which may indicate high arousal. Our RHMI helped drivers stay alert and respond to emergency LTA disconnection; however, some drivers showed a quick and large torque response only with RHMI-P.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Trübswetter, N., Bengler, K.: Why should I use ADAS? Advanced driver assistance systems and the elderly: knowledge, experience and usage barriers. In: 7th International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design (2013)

    Google Scholar 

  2. Viktorová, L., Šucha, M.: Drivers’ acceptance of advanced driver assistance systems–what to consider. Int. J. Traffic Transp. Eng. 8(3), 320–333 (2018)

    Article  Google Scholar 

  3. Jammes, Y., Behr, M., Llari, M., Bonicel, S., Weber, J.P., Berdah, S.: Emergency braking is affected by the use of cruise control. Traffic Injury Prev. 18(6), 636–641 (2017)

    Article  Google Scholar 

  4. Bruyas, M.P., Le Breton, B., Pauzié, A.: Ergonomic guidelines for the design of pictorial information. Int. J. Ind. Ergon. 21(5), 407–413 (1998)

    Article  Google Scholar 

  5. Karatas, N., et al.: Evaluation of AR-HUD interface during an automated intervention in manual driving. In: 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 2158–2164. IEEE (2020)

    Google Scholar 

  6. Langlois, S., Soualmi, B.: Augmented reality versus classical HUD to take over from automated driving: an aid to smooth reactions and to anticipate maneuvers. In: 2016 IEEE 19th International Conference on Intelligent Transportation Systems, ITSC, pp. 1571–1578. IEEE (2016)

    Google Scholar 

  7. Braun, M., Mainz, A., Chadowitz, R., Pfleging, B., Alt, F.: At your service: designing voice assistant personalities to improve automotive user interfaces. In: 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–11 (2019)

    Google Scholar 

  8. Serpell, J.: Anthropomorphism and anthropomorphic selection—Beyond the “cute response.” Soc. Anim. 11(1), 83–100 (2003)

    Article  Google Scholar 

  9. Tanaka, T., Fujikake, K., Yoshihara, Y., Karatas, N., Aoki, H., Kanamori, H.: Preliminary study for feasibility of driver agent in actual car environment—Driver agent for encouraging safe driving behavior. J. Transp. Technol. 10(02), 128 (2020)

    Google Scholar 

  10. Biocca, F., Harms, C., Burgoon, J.K.: Toward a more robust theory and measure of social presence: Review and suggested criteria. Presence: Teleoper. Virtual Environ. 12(5), 456–480 (2003)

    Article  Google Scholar 

  11. Bartneck, C., Forlizzi, J.: A design-centred framework for social human-robot interaction. In: RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication, IEEE Catalog No. 04TH8759, pp. 591–594. IEEE (2004)

    Google Scholar 

  12. Fong, T., Nourbakhsh, I., Dautenhahn, K.: A survey of socially interactive robots. Robot. Auton. Syst. 42(3–4), 143–166 (2003)

    Article  Google Scholar 

  13. Wang, M., Lee, S.C., Kamalesh Sanghavi, H., Eskew, M., Zhou, B., Jeon, M.: In-vehicle intelligent agents in fully autonomous driving: the effects of speech style and embodiment together and separately. In: 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 247–254 (2021)

    Google Scholar 

  14. Tanaka, T., et al.: Effect of difference in form of driving support agent to driver’s acceptability—Driver agent for encouraging safe driving behavior. J. Transp. Technol. 8(03), 194 (2018)

    Google Scholar 

  15. Anderson-Bashan, L., et al.: The greeting machine: an abstract robotic object for opening encounters. In: 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 595–602. IEEE (2018)

    Google Scholar 

  16. Parlitz, C., Hägele, M., Klein, P., Seifert, J., Dautenhahn, K.: Care-O-bot 3-rationale for human-robot interaction design. In: 39th International Symposium on Robotics, ISR, pp. 275–280 (2008)

    Google Scholar 

  17. Karatas, N., Yoshikawa, S., De Silva, P.R.S., Okada, M.: NAMIDA: multiparty conversation based driving agents in futuristic vehicle. In: Kurosu, M. (ed.) HCI 2015. LNCS, vol. 9171, pp. 198–207. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21006-3_20

    Chapter  Google Scholar 

  18. Zihsler, J., et al.: Carvatar: increasing trust in highly-automated driving through social cues. In: Adjunct Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 9–14 (2016)

    Google Scholar 

  19. Williams, K., Breazeal, C.: Reducing driver task load and promoting sociability through an affective intelligent driving agent (AIDA). In: Kotzé, P., Marsden, G., Lindgaard, G., Wesson, J., Winckler, M. (eds.) INTERACT 2013. LNCS, vol. 8120, pp. 619–626. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40498-6_53

    Chapter  Google Scholar 

  20. Ferrari, M., Quaresima, V.: A brief review on the history of human functional near-infrared spectroscopy (fNIRS) development and fields of application. Neuroimage 63(2), 921–935 (2012)

    Article  Google Scholar 

  21. Hirth, C., et al.: Noninvasive functional mapping of the human motor cortex using near-infrared spectroscopy. NeuroReport 7(12), 1977–1981 (1996)

    Article  Google Scholar 

  22. Hock, C., Müller-Spahn, F., Schuh-Hofer, S., Hofmann, M., Dirnagl, U., Villringer, A.: Age dependency of changes in cerebral hemoglobin oxygenation during brain activation: a near-infrared spectroscopy study. J. Cereb. Blood Flow Metab. 15(6), 1103–1108 (1995)

    Article  Google Scholar 

  23. Hoshi, Y., Tamura, M.: Detection of dynamic changes in cerebral oxygenation coupled to neuronal function during mental work in man. Neurosci. Lett. 150(1), 5–8 (1993)

    Article  Google Scholar 

  24. Obrig, H., et al.: Near-infrared spectroscopy: does it function in functional activation studies of the adult brain? Int. J. Psychophysiol. 35(2–3), 125–142 (2000)

    Article  Google Scholar 

  25. Villringer, A., Planck, J., Hock, C., Schleinkofer, L., Dirnagl, U.: Near infrared spectroscopy (NIRS): a new tool to study hemodynamic changes during activation of brain function in human adults. Neurosci. Lett. 154(1–2), 101–104 (1993)

    Article  Google Scholar 

  26. Watanabe, A., Kato, T.: Cerebrovascular response to cognitive tasks in patients with schizophrenia measured by near-infrared spectroscopy. Schizophr. Bull. 30(2), 435–444 (2004)

    Article  Google Scholar 

  27. Miyai, I., et al.: Cortical mapping of gait in humans: a near-infrared spectroscopic topography study. Neuroimage 14(5), 1186–1192 (2001)

    Article  Google Scholar 

  28. Okamoto, M., et al.: Multimodal assessment of cortical activation during apple peeling by NIRS and fMRI. Neuroimage 21(4), 1275–1288 (2004)

    Article  Google Scholar 

  29. Doi, H., Nishitani, S., Shinohara, K.: NIRS as a tool for assaying emotional function in the prefrontal cortex. Front. Hum. Neurosci. 7, 770 (2013)

    Article  Google Scholar 

  30. Harmon-Jones, E., Allen, J.J.: Anger and frontal brain activity: EEG asymmetry consistent with approach motivation despite negative affective valence. J. Pers. Soc. Psychol. 74(5), 1310 (1998)

    Article  Google Scholar 

  31. Kubo, K., Okanoya, K., Kawai, N.: Apology isn’t good enough: an apology suppresses an approach motivation but not the physiological and psychological anger. PLoS ONE 7(3), e33006 (2012)

    Article  Google Scholar 

  32. Nishikawa, T., Watanuki, K., Kaede, K., Muramatsu, K., Mashiko, N.: Effects of subjective visual fatigue on brain function during luminescent sentence reading task. In: 2020 IEEE/SICE International Symposium on System Integration (SII), pp. 390–394. IEEE (2020)

    Google Scholar 

  33. Suda, M., et al.: Decreased cortical reactivity underlies subjective daytime light sleepiness in healthy subjects: a multichannel near-infrared spectroscopy study. Neurosci. Res. 60(3), 319–326 (2008)

    Article  Google Scholar 

  34. Toyota Europe Homepage (2016). https://myportalcontent.toyota-europe.com/Manuals/toyota/C-HR_HV_OM_Europe_OM10538E.pdf

  35. Toyota USA, Detailed System Overview Lane Tracing Assist (LTA) (2019). https://www.youtube.com/watch?v=COfzwGQ9r7w&ab_channel=ToyotaUSA

  36. Tremoulet, P.D., Feldman, J.: Perception of animacy from the motion of a single object. Perception 29(8), 943–951 (2000)

    Article  Google Scholar 

  37. Goldstein, K.: Some experimental observations concerning the influence of colors on the function of the organism. Occup. Ther. 21(3), 147–151 (1942)

    Google Scholar 

  38. Küller, R., Mikellides, B., Janssens, J.: Color, arousal, and performance—A comparison of three experiments. Color Res. Appl. 34(2), 141–152 (2009)

    Article  Google Scholar 

  39. Fitts, P.M.: The information capacity of the human motor system in controlling the amplitude of movement. J. Exp. Psychol. 47(6), 381–391 (1954)

    Article  Google Scholar 

  40. Woodworth, R.S.: The accuracy of voluntary movement. Psychol. Rev.: Monogr. Suppl. 3(3), i–114 (1899)

    Google Scholar 

  41. Pinti, P., Cardone, D., Merla, A.: Simultaneous fNIRS and thermal infrared imaging during cognitive task reveal autonomic correlates of prefrontal cortex activity. Sci. Rep. 5(1), 1–14 (2015)

    Article  Google Scholar 

  42. Meyer, W.U., Niepel, M., Rudolph, U., Schützwohl, A.: An experimental analysis of surprise. Cogn. Emot. 5(4), 295–311 (1991)

    Article  Google Scholar 

  43. Yao, H., An, S., Zhou, H., Itoh, M.: Driver takeover performance in conditionally automated driving: sudden system failure situation versus ODD exit situation. SICE J. Control Meas. Syst. Integr. 14(1), 89–96 (2021)

    Article  Google Scholar 

  44. Vogelpohl, T., Kühn, M., Hummel, T., Vollrath, M.: Asleep at the automated wheel—Sleepiness and fatigue during highly automated driving. Accid. Anal. Prev. 126, 70–84 (2019)

    Article  Google Scholar 

  45. Schömig, N., Hargutt, V., Neukum, A., Petermann-Stock, I., Othersen, I.: The interaction between highly automated driving and the development of drowsiness. Proc. Manuf. 3, 6652–6659 (2015)

    Google Scholar 

  46. Karatas, N., et al.: Effects of including a robotic-human-machine interface and its human-likeness level in advanced driver assistance systems. Front. Robot. AI. (in revision)

    Google Scholar 

  47. Xu, Y., He, W.: More information = less aggression? Impact of information asymmetry on Chinese patients’ aggression. Front. Public Health 7, 118 (2019)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Nagoya University, Furo-cho, Chikusa-ku, Nagoya, Aichi, 464-8601, Japan

    Hiroko Tanabe, Yuki Yoshihara, Nihan Karatas & Takahiro Tanaka

  2. Chukyo University, 101-2, Yagotohonmachi, Showa-ku, Nagoya, Aichi, 466-8666, Japan

    Kazuhiro Fujikake

  3. Tokai Rika, Co., Ltd., Oguchi-cho, Niwa-gun, Aichi, 480-0195, Japan

    Shuhei Takeuchi, Tsuneyuki Yamamoto, Makoto Harazawa & Naoki Kamiya

Authors
  1. Hiroko Tanabe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuki Yoshihara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nihan Karatas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kazuhiro Fujikake
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Takahiro Tanaka
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shuhei Takeuchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tsuneyuki Yamamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Makoto Harazawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Naoki Kamiya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hiroko Tanabe .

Editor information

Editors and Affiliations

  1. Coventry University, Coventry, UK

    Don Harris

  2. Cranfield University, Cranfield, UK

    Wen-Chin Li

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tanabe, H. et al. (2022). Effects of a Robot Human-Machine Interface on Emergency Steering Control and Prefrontal Cortex Activation in Automatic Driving. In: Harris, D., Li, WC. (eds) Engineering Psychology and Cognitive Ergonomics. HCII 2022. Lecture Notes in Computer Science(), vol 13307. Springer, Cham. https://doi.org/10.1007/978-3-031-06086-1_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06086-1_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06085-4

  • Online ISBN: 978-3-031-06086-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics