Abstract
Complex traffic areas and high cognitive workload while driving are leading contributors to traffic crashes. Even though cognitive workload and stress have been previously assessed through various neurophysiological responses, they are rarely characterized simultaneously, limiting the triangulation of behavioral metrics (like drivers’ visual attention and facial coding) with physiological measures to investigate their interplay. The aim of the present study was to systematically characterize stress and cognitive workload through a multimodal assessment comprising eye-tracking, facial expressions, galvanic skin response (GSR), electromyography (EMG), electrocardiography (ECG) and respiration in three controlled driving simulations of varying complexity: 1. Baseline driving on an open road (Baseline); 2. Navigating between traffic cones (Cones); and 3. Driving in a neighborhood with multiple stressors (Traffic). The selected metrics were eye tracking dwell time, the presence of facial brow furrow, GSR peaks/minute, EMG activity of the upper trapezius muscle, heart rate and heart rate variability (HRV) and respiration cycles/minute. Physiological responses showed significant increases in GSR, heart rate and trapezius EMG activity with Cones and Traffic compared to Baseline. Eye tracking metrics were shown to be indicative of driving behavior in different conditions. There were no significant differences in facial expressions, HRV or respiration. These results are somewhat consistent with previous literature, suggesting that a multimodal approach to physiological signals can characterize affective and cognitive states in driving scenarios.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yang, X., et al.: The effects of traveling in different transport modes on galvanic skin response (GSR) as a measure of stress: an observational study. Environ. Int. 156, 106764 (2021). https://doi.org/10.1016/j.envint.2021.106764
Mehler B., Reimer, B., Wang, Y.: A comparison of the heart rate and heart rate variability indices in distinguishing single-task driving and driving under secondary cognitive workload. In: Proceedings of the Sixth International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design, pp. 590–597. Lake Tahoe, CA (2011)
Brookhuis, K.A., de Waard, D.: Monitoring drivers’ mental workload in driving simulators using physiological measures. Accid. Anal. Prev. 42, 898–903 (2010). https://doi.org/10.1016/j.aap.2009.06.001
Lenneman, J.K., Backs, R.W.: Cardiac autonomic control during simulated driving with a concurrent verbal working memory task. Hum. Factors 51, 404–418 (2009). https://doi.org/10.1177/0018720809337716
Lenneman, J.K., Backs, R.W.: Enhancing assessment of in-vehicle technology attention demands with cardiac measures. In: Proceedings of the 2nd International Conference on Automotive User Interfaces and Interactive Vehicular applications, pp. 20–21. ACM, New York (2010)
Collet, C., Clarion, A., Morel, M., Chapon, A., Petit, C.: Physiological and behavioral changes associated to the management of secondary tasks while driving. App. Ergon. 40(6), 1041–1046 (2009). https://doi.org/10.1016/j.apergo.2009.01.007
Boucsein, W.: Electrodemal Activity. 2nd edn. Springer, New York (2012). https://doi.org/10.1007/978-1-4614-1126-0
Mehler, B., Reimer, B., Coughlin, J.F., Dusek, J.A.: Impact of incremental increases in cognitive workload on physiological arousal and performance in young adult drivers. Transp. Res. Rec. 2138(1), 6–12 (2009). https://doi.org/10.3141/2138-02
Jabon, M., Bailenson, J., Pontikakis, E., Takayama, L., Nass, C.: Facial expression analysis for predicting unsafe driving behavior. IEEE Pervasive Comput. 10(4), 84–95 (2011). https://doi.org/10.1109/MPRV.2010.46
Strayer, D.L., Drews, F.A., Johnston, W.A.: Cell phone-induced failures of visual attention during simulated driving. J. Exp. Psychol. Appl. 9(1), 23–32 (2003). https://doi.org/10.1037/1076-898x.9.1.23
Liang, N., et al.: Using eye-tracking to investigate the effects of pre-takeover visual engagement on situation awareness during automated driving. Accid. Anal. Prev. 157, 106143 (2021). https://doi.org/10.1016/j.aap.2021.106143
Healey, J.A., Picard, R.W.: Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans. Intell. Transp. Syst. 6(2), 156–166 (2005). https://doi.org/10.1109/TITS.2005.848368
Fatisson, J., Oswald, V., Lalonde, F.: Influence diagram of physiological and environmental factors affecting heart rate variability: an extended literature overview. Heart Int. 11(1), 32–40 (2016). https://doi.org/10.5301/heartint.5000232
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Minen, M., Gregoret, L., Seernani, D., Wilson, J. (2023). Systematic Evaluation of Driver’s Behavior: A Multimodal Biometric Study. In: Stephanidis, C., Antona, M., Ntoa, S., Salvendy, G. (eds) HCI International 2023 Posters. HCII 2023. Communications in Computer and Information Science, vol 1836. Springer, Cham. https://doi.org/10.1007/978-3-031-36004-6_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-36004-6_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36003-9
Online ISBN: 978-3-031-36004-6
eBook Packages: Computer ScienceComputer Science (R0)