Abstract
This study introduces a multimodal approach for enhancing the accuracy of Driver Monitoring Systems (DMS) in detecting driver distraction. By integrating data from vehicle control units with vision-based information, the research aims to address the limitations of current DMS. The experimental setup involves a driving simulator and advanced computer vision, deep learning technologies for facial expression recognition, and head rotation analysis. The findings suggest that combining various data types—behavioral, physiological, and emotional—can significantly improve DMS’s predictive capability. This research contributes to the development of more sophisticated, adaptive, and real-time systems for improving driver safety and advancing autonomous driving technologies.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Antony, M.M., Whenish, R.: Advanced Driver Assistance Systems (ADAS) in AKathiresh, M., Neelaveni, R. (eds) Automotive Embedded Systems. EAI/Springer Innovations in Communication and Computing. Springer, Cham, pp. 165–181 (2021). https://doi.org/10.1007/978-3-030-59897-6_9
Owens, J., Dingus, T., Guo, F., et al.: Prevalence of Drowsy-Driving Crashes: Estimates from a Large-Scale Naturalistic Driving Study. AAA Foundation for Traffic Safety, Washington, DC (2018)
Fitzharrris, M., Liu, S., Stephens, A.N., Lenné, M.G.: The relative importance of real-time in-cab and external feedback in managing fatigue in real-world commercial transport operations. Traffic Inj. Prev. 18(1), 71–78 (2017)
Ortega, J.D., Canas, P.N., Nieto, M., Otaegui, O., Salgado, L.: Challenges of large-scale multi-camera datasets for driver monitoring systems. Sensors 22(7), 2554 (2022)
Hasenjäger, M., Heckmann, M., Wersing, H.: A survey of personalization for advanced driver assistance systems. IEEE Trans. Intell. Veh. 5(2), 335–344 (2019)
Ceccacci, S., Mengoni, M., Andrea, G., Giraldi, L., Carbonara, G., Castellano, A., Montanari, R.: A preliminary investigation towards the application of facial expression analysis to enable an emotion-aware car interface. In: Universal Access in Human-Computer Interaction. Applications and Practice: 14th International Conference, UAHCI 2020, Held as Part of the 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, Part II 22 (pp. 504–517). Springer International Publishing (2020). https://doi.org/10.1007/978-3-030-49108-6_36
European Commission: Road safety thematic report – Serious injuries. European Road Safety Observatory. Brussels, European Commission, Directorate General for Transport (2021)
Gardony, A.L., Lindeman, R.W., Brunyé, T.T.: Eye-tracking for human-centered mixed reality: promises and challenges. In: Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR), (Vol. 11310, pp. 230–247). SPIE (Feb 2020)
Dehzangi, O., Sahu, V., Taherisadr, M., Galster, S.: Multi-modal system to detect on-the-road driver distraction. In: 2018 21st International Conference on Intelligent Transportation Systems (ITSC), (pp. 2191–2196). IEEE (Nov 2018)
Muhammad, K., Ullah, A., Lloret, J., Del Ser, J., de Albuquerque, V.H.C.: Deep learning for safe autonomous driving: Current challenges and future directions. IEEE Trans. Intell. Transp. Syst. 22(7), 4316–4336 (2020)
Daza, I.G., Bergasa, L.M., Bronte, S., Yebes, J.J., Almazán, J., Arroyo, R.: Fusion of optimized indicators from advanced driver assistance systems (adas) for driver drowsiness detection. Sensors 14, 1106–1131 (2014)
Sandberg, D., Akerstedt, T., Anund, A., Kecklund, G., Wahde, M.: Detecting driver sleepiness using optimized nonlinear combinations of sleepiness indicators. IEEE Trans. Intell. Transp. Syst. 12(1), 97–108 (2010)
Zhang, H., Wu, C., Huang, Z., Yan, X., Qiu, T.Z.: Sensitivity of lane position and steering angle measurements to driver fatigue. Transp. Res. Rec. 2585(1), 67–76 (2016)
Khandakar, A., et al.: Portable system for monitoring and controlling driver behavior and the use of a mobile phone while driving. Sensors 19(7), 1563 (2019)
Nemcova, A., et al.: Multimodal features for detection of driver stress and fatigue. IEEE Trans. Intell. Transp. Syst. 22(6) (2021)
Dinges, D.F.: An overview of sleepiness and accidents. J. Sleep Res. 4(2), 4–14 (1995)
Kircher, A., Uddman, M., Sandin, J.: Vehicle control and drowsiness. Statens väg-och transportforskningsinstitut (2002)
Bunji, A: Evaluation of Mental Condition on Drivers by Analysis of Heart Rate Variability. Journal of the Society of Automotive Engineers of Japan No. 9437601, 1994
Milardo, S., Rathore, P., Amorim, M., Fugiglando, U., Santi, P., Ratti, C.: Understanding drivers’ stress and interactions with vehicle systems through naturalistic data analysis. IEEE Trans. Intell. Transp. Syst. 23(9), 14570–14581 (2021)
Bergasa, L.M., Nuevo, J., Sotelo, M.A., Barea, R., Lopez, M.E.: Real-time system for monitoring driver vigilance. IEEE Trans. Intell. Transport. Syst. 7(1), 63–77 (2006)
Çetinkaya, M., Acarman, T.: Driver impairment detection using decision tree based feature selection and classification. Results Eng. 18, 101025 (2023)
Wylie, C.D., Shultz, T., Miller, J.C., Mitler, M.M., Mackie, R.R.: Commercial motor vehicle driver fatigue and alertness study: Technical summary (1996)
Doudou, M., Bouabdallah, A., Berge-Cherfaoui, V.: Driver drowsiness measurement technologies: current research, market solutions, and challenges. Int. J. ITS Res. 18, 297–319 (2020)
Sagberg, F.: Road accidents caused by drivers falling asleep. Accid. Anal. Prev. 31, 639–649 (1999)
Verwey, W.B., Zaidel, D.: Predicting Drowsiness Accidents from Personal Attributes, Eye Blinks, and Ongoing Driving Behaviour. Report TM-97-B009. Soesterberg. TNO Human Factors Research Institute, The Netherlands (1997)
Aksjonov, A., Nedoma, P., Vodovozov, V., Petlenkov, E., Herrmann, M.: Detection and evaluation of driver distraction using machine learning and fuzzy logic. IEEE Trans. Intell. Transp. Syst. 20(6), 2048–2059 (2018)
Bittner, R., Hána, K., Poušek, L., Smrka, P., Schreib, P., Vysoký, P.: Detecting of Fatigue States of a Car Driver. In: Brause, R.W., Hanisch, E. (eds.) ISMDA 2000. LNCS, vol. 1933, pp. 260–273. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-39949-6_32
Sârbescu, P.: Aggressive driving in Romania: psychometric properties of the driving anger expression inventory. Transport. Res. F: Traffic Psychol. Behav. 15(5), 556–564 (2012)
Wu, Y.L., Tsai, H.Y., Huang, Y.C., Chen, B.H.: Accurate emotion recognition for driving risk prevention in driver monitoring system. In: 2018 IEEE 7TH Global Conference on Consumer Electronics (GCCE) (pp. 796–797). IEEE (Oct 2018)
Talipu, A., Generosi, A., Mengoni, M., Giraldi, L.: Evaluation of deep convolutional neural network architectures for emotion recognition in the wild. In: 2019 IEEE 23rd International Symposium on Consumer Technologies, pp. 25–27, 2019. IEEEhttps://doi.org/10.1109/ISCE.2019.8900994
Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops (2010). https://doi.org/10.1109/CVPRW.2010.5543262
Barsoum, E., Zhang, C., Ferrer, C.-C., Zhang, Z.:“Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distribution” (2016). https://doi.org/10.1145/2993148.2993165
Mollahosseini, A., Hasani, B., Mahoor, M.H.: Affectnet: a database for facial expression, valence, and arousal computing in the wild. IEEE Trans. Affect. Comput. 10(1), 18–31 (2017)
Feldman Barrett, L., Russell, J.A.: Independence and bipolarity in the structure of current affect. J. Pers. Soc. Psychol. 74(4), 967 (1998)
Ceccacci, S., Generosi, A., Cimini, G., Faggiano, S., Giraldi, L., Mengoni, M.: Facial coding as a mean to enable continuous monitoring of student’s behavior in e-Learning. In: teleXbe (Jan 2021)
Mallick, S.: Head Pose Estimation using OpenCV and Dlib | LearnOpenCV #. LearnOpenCV – Learn OpenCV, PyTorch, Keras, Tensorflow With Examples and Tutorials (May 2021). https://learnopencv.com/head-pose-estimation-using-opencv-and-dlib/
Soukupova, T., Cech, J.: “Eye Blink Detection Using Facial Landmarks.“ 21st Computer Vision Winter Workshop, Rimske Toplice, Slovenia (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Generosi, A., Villafan, J.Y., Montanari, R., Mengoni, M. (2024). A Multimodal Approach to Understand Driver’s Distraction for DMS. In: Antona, M., Stephanidis, C. (eds) Universal Access in Human-Computer Interaction. HCII 2024. Lecture Notes in Computer Science, vol 14696. Springer, Cham. https://doi.org/10.1007/978-3-031-60875-9_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-60875-9_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-60874-2
Online ISBN: 978-3-031-60875-9
eBook Packages: Computer ScienceComputer Science (R0)