Abstract
Although the driver’s emotion has been studied in the different driving environments (such as city and highway), understanding what eye metrics and facial expressions correspond to specific emotion and behavior based on subjective and biosensor data to study emotion in depth is not well researched in previous studies. Using an eye-integrated human-in-the-loop (HTIL) simulation experiment, we studied how drivers’ facial expressions and ocular measurements relate to emotions. We found that the driving environment could significantly affect drivers’ emotions, which is evident in their facial expressions and eye metrics data. In addition, such outcomes provide knowledge to human-computer-interaction (HCI) practitioners on designing emotion recognition systems in cars to have a robust understanding of the drivers’ emotions and help progress multimodal emotion recognition.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tavakoli, A., Balali, V., Heydarian, A.: A Multimodal Approach for Monitoring Driving Behavior and Emotions (2020)
Kleinginna, P.R., Kleinginna, A.M.: A categorized list of motivation definitions, with a suggestion for a consensual definition. Motiv. Emot. 5(3), 263–291 (1981). https://doi.org/10.1007/BF00993889
Pittermann, J., Pittermann, A., Minker, W.: Emotion recognition and adaptation in spoken dialogue systems. Int. J. Speech Technol. 13(1), 49–60 (2010). https://doi.org/10.1007/s10772-010-9068-y
Leahu, L., Schwenk, S., Sengers, P.: Subjective objectivity: negotiating emotional meaning. In: Proceedings of the 7th ACM Conference on Designing Interactive Systems, pp. 425–434 (2008)
Picard, R.W., Klein, J.: Computers that recognise and respond to user emotion: theoretical and practical implications. Interact. Comput. 14(2), 141–169 (2002)
Cohn, J.F.: Foundations of human computing: facial expression and emotion. In: Proceedings of the 8th International Conference on Multimodal Interfaces, pp. 233–238 (2006)
Martis, J.E.: Effective emotion recognition of expressions from facial features. 5(06), 4–7 (2017)
Geiger, A., Brandenburg, E., Stark, R.: Natural virtual reality user interface to define assembly sequences for digital human models. Appl. Syst. Innov. 3(1), 15 (2020)
Sedenberg, E., Wong, R., Chuang, J.: A window into the soul: biosensing in public. arXiv preprint arXiv:1702.04235 (2017)
Liu, L., et al.: Deep learning for generic object detection: a survey. arXiv 2018. arXiv preprint arXiv:1809.02165 (2019)
Martinez-Conde, S., Macknik, S.L., Hubel, D.H.: The role of fixational eye movements in visual perception. Nat. Rev. Neurosci. 5(3), 229–240 (2004)
Baujon, J., Basset, M., Gissinger, G.L.: Visual behaviour analysis and driver cognitive model. In: Proceedings of the 3rd IFAC Workshop on Advances in Automotive Control, Karlsruhe, Germany, pp. 47–52 (2001)
Brackstone, M., Waterson, B.: Are we looking where we are going? An exploratory examination of eye movement in high-speed driving. In: Proceedings of the 83rd Transportation Research Board Annual Meeting, vol. 2, p. 602 (2004)
Yan, Y., Yuan, H., Wang, X., Xu, T., Liu, H.: Study on driver’s fixation variation at entrance and inside sections of tunnel on highway. Adv. Mech. Eng. 7(1), 273427 (2015)
Nadal, M., Munar, E., Marty, G., Cela-Conde, C.J.: Visual complexity and beauty appreciation: explaining the divergence of results. Empirical Stud. Arts 28(2), 173–191 (2010)
Chan, M., Singhal, A.: Emotion matters: Implications for distracted driving. Saf. Sci. 72, 302–309 (2015)
Zhang, W., Zhang, X., Feng, Z., Liu, J., Zhou, M., Wang, K.: The fitness-to-drive of shift-work taxi drivers with obstructive sleep apnea: an investigation of self-reported driver behavior and skill. Transp. Res. Part F: Traffic Psychol. Behav. 59, 545–554 (2018)
Eherenfreund-Hager, A., Taubman-Ben-Ari, O., Toledo, T., Farah, H.: The effect of positive and negative emotions on young drivers a simulator study. Transp. Res. Part F: Traffic Psychol. Behav. 49, 236–243 (2017)
Du, N., et al.: Examining the effects of emotional valence and arousal on takeover performance in conditionally automated driving. Transp. Res. Part C: Emerg. Technol. 112, 78–87 (2020)
Hedlund, J., Simpson, H.M., Mayhew, D.R.: Summary of proceedings and recommendations. In: International Conference on Distracted Driving. The Traffic Injury Research Foundation, The Canadian Automobile Association, Ottawa (2006)
Nyström, M., Holmqvist, K.: An adaptive algorithm for fixation, saccade, and glissade detection in eyetracking data. Behav. Res. Methods 42(1), 188–204 (2010). https://doi.org/10.3758/BRM.42.1.188
Ekman, P., Friesen, W.V.: Facial Action Coding System: Investigator’s Guide. Consulting Psychologists Press, Palo Alto (1978)
Steyer, R., Schwenkmezger, P., Notz, P., Eid, M.: MDMQ questionnaire (English version of MDBF). Jena: Friedrich-Schiller-Universität Jena, Institut für Psychologie, Lehrstuhl für Methodenlehre und Evaluationsforschung (2014). https://www.metheval.uni-jena.de/mdbf.php. Accessed 4 Apr 2016
Meinlschmidt, G., et al.: Smartphone-based psychotherapeutic micro-interventions to improve mood in a real-world setting. Front. Psychol. 7, 1112 (2016)
Hinz, A., Daig, I., Petrowski, K., Brähler, E.: Die stimmung in der deutschen bevölkerung: referenzwerte für den mehrdimensionalen befindlichkeitsfragebogen MDBF. PPmP-Psychother. Psychosom. Med. Psychol. 62(02), 52–57 (2012)
Park, J., Abdel-Aty, M., Yina, W., Mattei, I.: Enhancing in-vehicle driving assistance information under connected vehicle environment. IEEE Trans. Intell. Transp. Syst. 20(9), 3558–3567 (2018)
Lynch, B.K.: Designing qualitative research by catherine marshall an Gretchen B. Rossman. Issues Appl. Linguist. 1(2), 1–9 (1990)
Schutte, N.S., Malouff, J.M., Thorsteinsson, E.B., Bhullar, N., Rooke, S.E.: A meta-analytic investigation of the relationship between emotional intelligence and health. Pers. Individ. Differ. 42(6), 921–933 (2007)
Guarnera, M., Hichy, Z., Cascio, M.I., Carrubba, S.: Facial expressions and ability to recognize emotions from eyes or mouth in children. Eur. J. Psychol. 11(2), 183 (2015)
Li, J., Jin, K., Zhou, D., Kubota, N., Zhaojie, J.: Attention mechanism-based CNN for facial expression recognition. Neurocomputing 411, 340–350 (2020)
Hassib, M., Braun, M., Pfleging, B., Alt, F.: Detecting and influencing driver emotions using psycho-physiological sensors and ambient light. In: Lamas, D., Loizides, F., Nacke, L., Petrie, H., Winckler, M., Zaphiris, P. (eds.) Human-Computer Interaction – INTERACT 2019, vol. 11746, pp. 721–742. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29381-9_43
Mesken, J.: Determinants and consequences of drivers’ emotions. Stichting Wetenschappelijk Onderzoek Verkeersveiligheid SWOV (2006)
Remington, R.W.: Attention and saccadic eye movements. J. Exp. Psychol.: Hum. Percept. Perform. 6(4), 726 (1980)
Murphy-Chutorian, E., Trivedi, M.M.: Head pose estimation and augmented reality tracking: an integrated system and evaluation for monitoring driver awareness. IEEE Trans. Intell. Transp. Syst. 11(2), 300–311 (2010)
Sahayadhas, A., Sundaraj, K., Murugappan, M.: Detecting driver drowsiness based on sensors: a review. Sensors 12(12), 16937–16953 (2012)
Mashko, A.: Subjective methods for assessment of driver drowsiness. Acta Polytech. CTU Proc. 12, 64–67 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mostowfi, S., Kim, J.H. (2022). Understanding Drivers’ Physiological Responses in Different Road Conditions. In: Krömker, H. (eds) HCI in Mobility, Transport, and Automotive Systems. HCII 2022. Lecture Notes in Computer Science, vol 13335. Springer, Cham. https://doi.org/10.1007/978-3-031-04987-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-04987-3_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-04986-6
Online ISBN: 978-3-031-04987-3
eBook Packages: Computer ScienceComputer Science (R0)