research-article

Multimodal Car Driver Stress Recognition

Authors:

Paolo Napoletano,

Raimondo SchettiniAuthors Info & Claims

PervasiveHealth'19: Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare

Pages 302 - 307

https://doi.org/10.1145/3329189.3329221

Published: 20 May 2019 Publication History

Abstract

In this paper we address the problem of multimodal car driver stress recognition. To this aim, four different signals are considered: heart rate (HR), breathing rate (BR), palm EDA (P-EDA), and perinasal perspitation (PER-EDA). The raw signals are windowed and for each window 21 different features, including both time-domain and frequency-domain descriptors, are extracted. The recognition problem is formulated as a stress vs no-stress binary problem, and is addressed in two different experimental setups: five-fold cross validation and leave one subject out. In both setups the extracted features are classified, both individually and concatenated, with three different classifiers (k--NN, SVM, and ANN) using them both alone and stacking their predictions. Experiments run on a publicly available database of multimodal signals acquired in a controlled experiment on a driving simulator show that the best recognition results are obtained feeding the classifiers with the concatenation of the features of all the signals considered, reaching a micro average accuracy of 77.25% and 65.09% in the two experimental setups respectively.

References

[1]

Shahina Begum. 2013. Intelligent driver monitoring systems based on physiological sensor signals: A review. In Intelligent Transportation Systems-(ITSC), 2013 16th International IEEE Conference on. IEEE, 282--289.

[2]

Simone Bianco, Remi Cadene, Luigi Celona, and Paolo Napoletano. 2018. Benchmark Analysis of Representative Deep Neural Network Architectures. IEEE Access 6, 1 (2018), 64270--64277.

[3]

Youjun Choi, Sang Ik Han, Seung-Hyun Kong, and Hyunwoo Ko. 2016. Driver status monitoring systems for smart vehicles using physiological sensors: A safety enhancement system from automobile manufacturers. IEEE Signal Processing Magazine 33, 6 (2016), 22--34.

[4]

Dan Conway, Ian Dick, Zhidong Li, Yang Wang, and Fang Chen. 2013. The effect of stress on cognitive load measurement. In IFIP Conference on Human-Computer Interaction. Springer, 659--666.

Digital Library

[5]

Yanchao Dong, Zhencheng Hu, Keiichi Uchimura, and Nobuki Murayama. 2011. Driver inattention monitoring system for intelligent vehicles: A review. IEEE transactions on intelligent transportation systems 12, 2 (2011), 596--614.

Digital Library

[6]

Marta Gonçalves, Roberto Amici, Raquel Lucas, Torbjörn Åkerstedt, Fabio Cirignotta, Jim Horne, Damien Léger, Walter T McNicholas, Markku Partinen, Joaquín Téran-Santos, et al. 2015. Sleepiness at the wheel across Europe: a survey of 19 countries. Journal of sleep research 24, 3 (2015), 242--253.

[7]

Jennifer A Healey and Rosalind W Picard. 2005. Detecting stress during real-world driving tasks using physiological sensors. IEEE Transactions on intelligent transportation systems 6, 2 (2005), 156--166.

Digital Library

[8]

Donald L Hendricks, James C Fell, Mark Freedman, and JF Page. 2001. The relative frequency of unsafe driving acts in serious traffic crashes. National Highway Traffic Safety Administration (NHTSA), report DOT-HS-809-206, Washington, DC.

[9]

Antonio Lanatà, Gaetano Valenza, Alberto Greco, Claudio Gentili, Riccardo Bartolozzi, Francesco Bucchi, Francesco Frendo, and Enzo Pasquale Scilingo. 2015. How the autonomic nervous system and driving style change with incremental stressing conditions during simulated driving. IEEE Transactions on Intelligent Transportation Systems 16, 3 (2015), 1505--1517.

Digital Library

[10]

Cathy Macharis, Alain Verbeke, and Klaas De Brucker. 2004. The strategic evaluation of new technologies through multicriteria analysis: the ADVISORS case. Research in Transportation Economics 8 (2004), 443--462.

[11]

Gerald Matthews, Ryan Wohleber, Jinchao Lin, Gregory Funke, and Catherine Neubauer. 2019. Monitoring Task Fatigue in Contemporary and Future Vehicles: A Review. In Advances in Human Factors in Simulation and Modeling, Daniel N. Cassenti (Ed.). Springer International Publishing, Cham, 101--112.

[12]

Daniela Micucci, Marco Mobilio, Paolo Napoletano, and Francesco Tisato. 2017. Falls as anomalies? An experimental evaluation using smartphone accelerometer data. Journal of Ambient Intelligence and Humanized Computing 8, 1 (2017), 87--99.

[13]

Nermine Munla, Mohamad Khalil, Ahmad Shahin, and Azzam Mourad. 2015. Driver stress level detection using HRV analysis. In Advances in Biomedical Engineering (ICABME), 2015 International Conference on. IEEE, 61--64.

[14]

Mario Muñoz-Organero and Victor Corcoba-Magaña. 2017. Predicting Upcoming Values of Stress While Driving. IEEE Transactions on Intelligent Transportation Systems 18, 7 (2017), 1802--1811.

Digital Library

[15]

Paolo Napoletano and Stefano Rossi. 2018. Combining heart and breathing rate for car driver stress recognition. In 2018 IEEE 8th International Conference on Consumer Electronics-Berlin (ICCE-Berlin). IEEE, 1--5.

[16]

Simon Ollander, Christelle Godin, Sylvie Charbonnier, and Aurélie Campagne. 2016. Feature and Sensor Selection for Detection of Driver Stress. In PhyCS. 115--122.

[17]

Mohammad Naim Rastgoo, Bahareh Nakisa, Andry Rakotonirainy, Vinod Chandran, and Dian Tjondronegoro. 2018. A critical review of proactive detection of driver stress levels based on multimodal measurements. ACM Computing Surveys (CSUR) 51, 5 (2018), 88.

Digital Library

[18]

Mario Salai, István Vassányi, and István Kósa. 2016. Stress detection using low cost heart rate sensors. Journal of healthcare engineering 2016 (2016).

[19]

Dvijesh Shastri, Manos Papadakis, Panagiotis Tsiamyrtzis, Barbara Bass, Ioannis T Pavlidis, et al. 2012. Perinasal imaging of physiological stress and its affective potential. IEEE Trans. Affective Computing 3, 3 (2012), 366--378.

Digital Library

[20]

Salah Taamneh, Panagiotis Tsiamyrtzis, Malcolm Dcosta, Pradeep Buddharaju, Ashik Khatri, Michael Manser, Thomas Ferris, Robert Wunderlich, and Ioannis Pavlidis. 2017. A multimodal dataset for various forms of distracted driving. Scientific data 4 (2017), 170110.

[21]

Brian C Tefft et al. 2014. Prevalence of motor vehicle crashes involving drowsy drivers, United States, 2009-2013. Report Prepared for the American Automobile Association (AAA) Foundation for Traffic Safety Washington, DC.

[22]

R Vivoli, Margherita Bergomi, Sergio Rovesti, P Bussetti, and GM Guaitoli. 2006. Biological and behavioral factors affecting driving safety. Journal of preventive medicine and hygiene 47, 2 (2006).

[23]

Jie Xu, Yang Wang, Fang Chen, Ho Choi, Guanzhong Li, Siyuan Chen, and Sazzad Hussain. 2011. Pupillary response based cognitive workload index under luminance and emotional changes. In CHI'11 Extended Abstracts on Human Factors in Computing Systems. ACM, 1627--1632.

Digital Library

[24]

Roberto Zangróniz, Arturo Martínez-Rodrigo, José Manuel Pastor, María T López, and Antonio Fernández-Caballero. 2017. Electrodermal Activity Sensor for Classification of Calm/Distress Condition. Sensors 17, 10 (2017), 2324.

Cited By

Yasser FHatem SWaleed HAhmed ZHossam BAli RAtia A(2024)Driver Behavior Tracking: A Hierarchical Classification Approach2024 14th International Conference on Electrical Engineering (ICEENG)10.1109/ICEENG58856.2024.10566383(231-236)Online publication date: 21-May-2024
https://doi.org/10.1109/ICEENG58856.2024.10566383
Yasser FHatem SAhmed ZWaleed HHossam BAli RAtia A(2024)Driver and Vehicle Unsafe Behavior Tracking using Deep Learning2024 6th International Conference on Computing and Informatics (ICCI)10.1109/ICCI61671.2024.10485085(75-82)Online publication date: 6-Mar-2024
https://doi.org/10.1109/ICCI61671.2024.10485085
Vaezi MNasri MAzimifar FMosleh M(2024)Driver Assistance System for Stress Recognition by Handcrafted Feature Extraction and Convolutional Neural Network2024 20th CSI International Symposium on Artificial Intelligence and Signal Processing (AISP)10.1109/AISP61396.2024.10475225(1-5)Online publication date: 21-Feb-2024
https://doi.org/10.1109/AISP61396.2024.10475225
Show More Cited By

Recommendations

Automatic stress detection in car drivers based on non-invasive physiological signals using machine learning techniques
Abstract
Stress is now thought to be a major cause to a wide range of human health issues. However, many people may ignore their stress feelings and disregard to take action before serious physiological and mental disorders take place. The heart rate (HR) ...
Applying neural network analysis on heart rate variability data to assess driver fatigue

Research highlights In this study we show heart rate variability which can be used as a passive means to quantify drowsiness. Frequency domain components of HRV is used for early detection of fatigue. We also designed a neural network based artificial ...
Estimation of driver's fatigue based on steering wheel angle
EPCE'11: Proceedings of the 9th international conference on Engineering psychology and cognitive ergonomics

Driver's fatigue has been verified as a major factor in many traffic accidents. The estimation of driver's vigilance by steering wheel angle is good way because it is a non-invasive method compared with EEG. An adaptive vigilance estimation methodology ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

PervasiveHealth'19: Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare

May 2019

475 pages

ISBN:9781450361262

DOI:10.1145/3329189

General Chairs:
Oscar Mayora
Fondazione Bruno Kessler, Italy
,
Stefano Forti
Fondazione Bruno Kessler, Italy
,
Program Chairs:
Jochen Meyer,
Lena Mamykina

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

EAI: The European Alliance for Innovation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

PervasiveHealth'19

PervasiveHealth'19: The 13th International Conference on Pervasive Computing Technologies for Healthcare

May 20 - 23, 2019

Trento, Italy

Acceptance Rates

Overall Acceptance Rate 55 of 116 submissions, 47%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
424
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)2

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yasser FHatem SWaleed HAhmed ZHossam BAli RAtia A(2024)Driver Behavior Tracking: A Hierarchical Classification Approach2024 14th International Conference on Electrical Engineering (ICEENG)10.1109/ICEENG58856.2024.10566383(231-236)Online publication date: 21-May-2024
https://doi.org/10.1109/ICEENG58856.2024.10566383
Yasser FHatem SAhmed ZWaleed HHossam BAli RAtia A(2024)Driver and Vehicle Unsafe Behavior Tracking using Deep Learning2024 6th International Conference on Computing and Informatics (ICCI)10.1109/ICCI61671.2024.10485085(75-82)Online publication date: 6-Mar-2024
https://doi.org/10.1109/ICCI61671.2024.10485085
Vaezi MNasri MAzimifar FMosleh M(2024)Driver Assistance System for Stress Recognition by Handcrafted Feature Extraction and Convolutional Neural Network2024 20th CSI International Symposium on Artificial Intelligence and Signal Processing (AISP)10.1109/AISP61396.2024.10475225(1-5)Online publication date: 21-Feb-2024
https://doi.org/10.1109/AISP61396.2024.10475225
Lee GLee S(2024)Deep-Learning Domain Adaptation to Improve Generalizability across Subjects and Contexts in Detecting Construction Workers’ Stress from BiosignalsJournal of Computing in Civil Engineering10.1061/JCCEE5.CPENG-566538:3Online publication date: May-2024
https://doi.org/10.1061/JCCEE5.CPENG-5665
Amin MUllah KAsif MShah HWaheed ADin I(2024)Fuzzy performance estimation of real-world driver’s stress recognition models based on physiological signals and deep learning approachJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-024-04834-7Online publication date: 3-Aug-2024
https://doi.org/10.1007/s12652-024-04834-7
Amin MUllah KAsif MShah HMehmood AKhan M(2023)Real-World Driver Stress Recognition and Diagnosis Based on Multimodal Deep Learning and Fuzzy EDAS ApproachesDiagnostics10.3390/diagnostics1311189713:11(1897)Online publication date: 29-May-2023
https://doi.org/10.3390/diagnostics13111897
Ye CLi WLi ZMaguluri GGrimble JBonatt JMiske JIftimia NLin SGrimm M(2022)Smart Steering Sleeve (S3): A Non-Intrusive and Integrative Sensing Platform for Driver Physiological MonitoringSensors10.3390/s2219729622:19(7296)Online publication date: 26-Sep-2022
https://doi.org/10.3390/s22197296
Elshafei MCosta DShihab E(2022)Toward the Personalization of Biceps Fatigue Detection Model for Gym Activity: An Approach to Utilize Wearables’ Data from the CrowdSensors10.3390/s2204145422:4(1454)Online publication date: 14-Feb-2022
https://doi.org/10.3390/s22041454
Saha SJindal KShakti DTewary SSardana V(2022) Chirplet transform‐based machine‐learning approach towards classification of cognitive state change using galvanic skin response and photoplethysmography signals Expert Systems10.1111/exsy.1295839:6Online publication date: 4-Mar-2022
https://doi.org/10.1111/exsy.12958
Amin MUllah KAsif MWaheed AHaq SZareei MBiswal R(2022)ECG-Based Driver’s Stress Detection Using Deep Transfer Learning and Fuzzy Logic ApproachesIEEE Access10.1109/ACCESS.2022.315865810(29788-29809)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3158658
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten