skip to main content
10.1145/3630970.3631059acmotherconferencesArticle/Chapter ViewAbstractPublication PagesclihcConference Proceedingsconference-collections
extended-abstract

Development of a Multimodal Model for Emotions Recognition in Drivers Using Convolutional Neural Networks

Published: 06 January 2024 Publication History

Abstract

This research project, conducted within a PhD program in Engineering Sciences at the Autonomous University of Zacatecas, Mexico, aims to develop and validate a multimodal model for emotion recognition in drivers using convolutional neural networks. The primary motivation is to enhance road safety by recognizing and understanding drivers’ emotional states, ultimately improving the driving experience. The study leverages affective computing and advanced driver assistance systems to achieve these goals. The research includes the generation of a facial geometry and motor activity database, development of classification models, implementation of convolutional neural networks, and performance assessment. Preliminary results have shown promising progress, and the work is ongoing. The expected contributions involve high-impact research and the potential to establish correlations between human behavior and emotions, with the possibility of real-time emotion recognition in drivers. Future work includes the incorporation of tracking pose data and deep learning techniques for more comprehensive emotion recognition.

References

[1]
2018. WHO: Road traffic injuries., [Online] pages. https://www.who.int/news-room/fact-sheets/detail/road-traffic-injuries
[2]
Michael Braun and Florian Alt. 2019. Affective Assistants: A Matter of States and Traits. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3290607.3313051
[3]
Rafael Calvo, Sidney D’Mello, Jonathan Gratch, and Arvid Kappas. 2015. 1Introduction to Affective Computing. In The Oxford Handbook of Affective Computing. Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199942237.013.040 arXiv:https://academic.oup.com/book/0/chapter/212003020/chapter-ag-pdf/44596649/book_28057_section_212003020.ag.pdf
[4]
Z. Cao, G. Hidalgo Martinez, T. Simon, S. Wei, and Y. A. Sheikh. 2019. OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence (2019).
[5]
Yaodong Cui, Yintao Ma, Wenbo Li, Ning Bian, Guofa Li, and Dongpu Cao. 2020. Multi-EmoNet: A Novel Multi-Task Neural Network for Driver Emotion Recognition. In IFAC-PapersOnLine, Vol. 53. 650–655. https://doi.org/10.1016/j.ifacol.2021.04.155
[6]
Satyajit Nayak, Bingi Nagesh, Aurobinda Routray, and Monalisa Sarma. 2021. A Human–Computer Interaction framework for emotion recognition through time-series thermal video sequences. Computers & Electrical Engineering 93 (2021), 107280. https://doi.org/10.1016/j.compeleceng.2021.107280
[7]
Geesung Oh, Junghwan Ryu, Euiseok Jeong, Ji Hyun Yang, Sungwook Hwang, Sangho Lee, and Sejoon Lim. 2021. Drer: Deep learning–based driver’s real emotion recognizer. Sensors 21, 6 (2021), 1–29. https://doi.org/10.3390/s21062166
[8]
Mrinalini Patil and S. Veni. 2019. Driver emotion recognition for enhancement of human machine interface in vehicles. In Proceedings of the 2019 IEEE International Conference on Communication and Signal Processing, ICCSP 2019. 0420–0424. https://doi.org/10.1109/ICCSP.2019.8698045
[9]
James A. Russell. 1980. A circumplex model of affect. Journal of Personality and Social Psychology 39, 6 (1980). https://doi.org/10.1037/h0077714
[10]
Sina Shafaei, Tahir Hacizade, and Alois Knoll. 2019. Integration of Driver Behavior into Emotion Recognition Systems: A Preliminary Study on Steering Wheel and Vehicle Acceleration. In Lecture Notes in Computer Science, Vol. 11367 LNCS. 386–409. https://doi.org/10.1007/978-3-030-21074-8_32
[11]
Bindu Verma and Ayesha Choudhary. 2018. A Framework for Driver Emotion Recognition using Deep Learning and Grassmann Manifolds. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, Vol. 21. 1421–1426. https://doi.org/10.1109/ITSC.2018.8569461
[12]
Bindu Verma and Ayesha Choudhary. 2018. Deep Learning Based Real-Time Driver Emotion Monitoring. In 2018 IEEE International Conference on Vehicular Electronics and Safety, ICVES 2018. 1–6. https://doi.org/10.1109/ICVES.2018.8519595
[13]
Jianhua Zhang, Zhong Yin, Peng Chen, and Stefano Nichele. 2020. Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review. Information Fusion 59 (2020), 103–126. https://doi.org/10.1016/j.inffus.2020.01.011

Index Terms

  1. Development of a Multimodal Model for Emotions Recognition in Drivers Using Convolutional Neural Networks

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    CLIHC '23: Proceedings of the XI Latin American Conference on Human Computer Interaction
    October 2023
    247 pages
    ISBN:9798400716577
    DOI:10.1145/3630970
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 06 January 2024

    Check for updates

    Author Tags

    1. ADAS
    2. affective computing
    3. behavior
    4. convolutional neural networks
    5. motor activity

    Qualifiers

    • Extended-abstract
    • Research
    • Refereed limited

    Conference

    CLIHC 2023

    Acceptance Rates

    Overall Acceptance Rate 14 of 42 submissions, 33%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 26
      Total Downloads
    • Downloads (Last 12 months)21
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media