skip to main content
10.1145/2390256.2390292acmotherconferencesArticle/Chapter ViewAbstractPublication PagesautomotiveuiConference Proceedingsconference-collections
research-article

"FaceLight": potentials and drawbacks of thermal imaging to infer driver stress

Published: 17 October 2012 Publication History

Abstract

Driving a modern vehicle is a complex, cognitive demanding task involving concentrated observation of the road, roadside, car status, information displays of assistance systems, etc. Drivers are conscious about this overabundance of information, nevertheless, they are operating tertiary controls, talking on the phone, smoking cigarettes, having lunch, reading maps or meeting agendas, or working on their computer. As a consequence -- caused by visual/manual/cognitive demand and limited multitasking capabilities-- precarious driving situations are created. Solutions are rare but badly needed to prevent imminent danger on the roads. To explore the potential of thermal imaging to infer mental conditions of the driver in an unobtrusive manner, and to use this information to automatically react to a detected risky state, we have developed the "FaceLight" prototype and performed a lab-based driving simulator study to evaluate the interface under conditions of varying workload. With "FaceLight" the driver can be interpreted as sort of signal light, with a 'red face' (hot surface temperature) standing for high stress or cognitive demand while a 'green face' (cooler temperature) equals to a more relaxed, stress-free mental state. Initial results have revealed that this technology has potential to capture shifts in the mental state of an individual in an inattentive manner, but highlighted also that a lot of influencing factors still need to be incorporated to reliably recognize a specific state solely based on facial skin temperature.

References

[1]
E. Angelopoulou. The reflectance spectrum of human skin. Technical report, Department of Computer and Information Science. University of Pennsylvania, 1999.
[2]
R. B. Barnes. Diagnostic thermography. Appl. Opt., 7(9):1673--1685, Sep 1968.
[3]
BMW Group. Bmw group driver assistance systems: More comfort, greater assurance, improved safety. Technical report, Bayerische Motoren Werke Aktiengesellschaft, Technology Communication, D-80788 München, August 2008. pp. 12.
[4]
J. K. Choi, K. Miki, S. Sagawa, and K. Shiraki. Evaluation of mean skin temperature formulas by infrared thermography. International Journal of Biometeorology, 41:68--75, 1997.
[5]
M. L. Cohen. Measurement of the thermal properties of human skin. a review. J Investig Dermatol, 69(3):333--338, Sept. 1977.
[6]
N. E. Day. Estimating the components of a mixture of normal distributions. Biometrika, 56(3):pp. 463--474, 1969.
[7]
A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the em algorithm. JOURNAL OF THE ROYAL STATISTICAL SOCIETY, SERIES B, 39(1):1--38, 1977.
[8]
H. Genno, K. Ishikawa, O. Kanbara, M. Kikumoto, Y. Fujiwara, R. Suzuk, and M. Osumi. Using facial skin temperature to objectively evaluate sensations. International Journal of Industrial Ergonomics, 19(2):161--171, 1997.
[9]
H. Genno, A. Saijo, H. Yoshida, R. Suzuki, and M. Osumi. Non-contact method for measuring facial skin temperature. International Journal of Industrial Ergonomics, 19(2):147--159, 1997.
[10]
D. M. Gronwall. Paced auditory serial-addition task: a measure of recovery from concussion. Perceptual and motor skills, 44(2):367--373, 1977.
[11]
J. L. Harbluk, J. S. Mitroi, and P. C. Burns. Three navigation systems with three tasks: Using the lane-change test (lct) to assess distraction demand. In Proceedings of the 5th International Driving Symposium on Human Factors in Driver Assessment and Design, pages 24--30. University of Iowa, Iowa City, 2009.
[12]
B. Hernández, G. Olague, R. Hammoud, L. Trujillo, and E. Romero. Visual learning of texture descriptors for facial expression recognition in thermal imagery. Comput. Vis. Image Underst., 106(2--3):258--269, May 2007.
[13]
V. Kastrinaki, M. Zervakis, and K. Kalaitzakis. A survey of video processing techniques for traffic applications. Image and Vision Computing, 21:359--381, 2003.
[14]
H. Kataoka, H. Kano, H. Yoshida, A. Saijo, M. Yasuda, and M. Osumi. Development of a skin temperature measuring system for non-contact stress evaluation. In Engineering in Medicine and Biology Society, 1998. Proceedings of the 20th Annual International Conference of the IEEE, volume 2, pages 940--943 vol.2, oct-1 nov 1998.
[15]
M. Korukcu and M. Kilic. Tracking hand and facial skin temperatures in an automobile by using ir-thermography during heating period. Gazi University Journal of Science, 25(1), 2012.
[16]
J. A. Levine, I. Pavlidis, and M. Cooper. The face of fear. Lancet, 357(7):1757, 2001.
[17]
Z. Liu and S. Wang. Emotion recognition using hidden markov models from facial temperature sequence. In Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part II, ACII'11, pages 240--247, Berlin, Heidelberg, 2011. Springer-Verlag.
[18]
M. Matscheko, A. Ferscha, A. Riener, and M. Lehner. Tactor placement in wrist worn wearables. In 14th annual IEEE International Symposium on Wearable Computers (ISWC'10), October 10--13, Seoul, South Korea, page 8. IEEE Computer Society Press, October 2010.
[19]
S. Mattes. The lane-change-task as a tool for driver distraction evaluation. Most, pages 1--4, 2003.
[20]
W. Minkina and S. Dudzik. Infrared Thermography: Errors and Uncertainties. John Wiley & Sons, 2009.
[21]
A. Nakayama. Thermal physiology. Science and Engineering Co., 1991.
[22]
C. K. Ora and V. G. Duffy. Development of a facial skin temperature-based methodology for non-intrusive mental workload measurement. Occupational Ergonomics, 7(2):83--94, 2007.
[23]
C. J. D. Patten. Cognitive Workload and the Driver: Understanding the Effects of Cognitive Workload on Driving from a Human Information Processing Perspective. PhD thesis, Dept. of Psychology, Stockholm University, Sweden, 2007.
[24]
D. A. Pollina, A. B. Dollins, S. M. Senter, T. E. Brown, I. Pavlidis, J. A. Levine, and A. H. Ryan. Facial skin surface temperature changes during a concealed information test. Annals of Biomedical Engineering, 34(7):1182--1189, 2006.
[25]
M. Rachel. Effects of stress on the heart. {online}, last retrieved June 10, 2012, May 2012.
[26]
M. L. Reyes, J. D. Lee, Y. Liang, J. D. Hoffman, and R. W. Huang. Capturing driver response to in-vehicle human-machine interface technologies using facial thermography. In Proceedings of the 5th International Driving Symposium on Human Factors in Driver Assessment and Design, volume 5, pages 536--542. University of Iowa, Iowa City, 2009.
[27]
M. A. Staal. Stress, cognition, and human performance: A literature review and conceptual framework. Processing, 212824(August):177, 2004.
[28]
P. Tsiamyrtzis, J. Dowdall, D. Shastri, I. T. Pavlidis, M. G. Frank, and P. Ekman. Imaging facial physiology for the detection of deceit. Int. J. Comput. Vision, 71(2):197--214, Feb. 2007.
[29]
S. Wang, Z. Liu, S. Lv, Y. Lv, G. Wu, P. Peng, F. Chen, and X. Wang. A natural visible and infrared facial expression database for expression recognition and emotion inference. Multimedia, IEEE Transactions on, 12(7):682--691, nov. 2010.
[30]
A. Wesley, D. Shastri, and I. Pavlidis. A novel method to monitor driver's distractions. In Proceedings of the 28th of the international conference extended abstracts on Human factors in computing systems, CHI EA '10, pages 4273--4278, New York, NY, USA, 2010. ACM.
[31]
Y. Yoshitomi. Facial expression recognition for speaker using thermal image processing and speech recognition system. In Proceedings of the 10th WSEAS international conference on Applied computer science, ACS'10, pages 182--186, Stevens Point, Wisconsin, USA, 2010. World Scientific and Engineering Academy and Society (WSEAS).

Cited By

View all
  • (2024)Application of machine learning techniques for driving errors analysis: systematic literature reviewInternational Journal of Crashworthiness10.1080/13588265.2023.230114629:5(785-793)Online publication date: 7-Jan-2024
  • (2023)A Survey on Measuring Cognitive Workload in Human-Computer InteractionACM Computing Surveys10.1145/358227255:13s(1-39)Online publication date: 13-Jul-2023
  • (2022)Towards Implicit Interaction in Highly Automated Vehicles - A Systematic Literature ReviewProceedings of the ACM on Human-Computer Interaction10.1145/35467266:MHCI(1-21)Online publication date: 20-Sep-2022
  • Show More Cited By

Index Terms

  1. "FaceLight": potentials and drawbacks of thermal imaging to infer driver stress

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      AutomotiveUI '12: Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications
      October 2012
      280 pages
      ISBN:9781450317511
      DOI:10.1145/2390256
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      In-Cooperation

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 17 October 2012

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. behavior detection
      2. driver stress
      3. physiological response
      4. thermal imaging

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      AutomotiveUI '12

      Acceptance Rates

      Overall Acceptance Rate 248 of 566 submissions, 44%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)9
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 10 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Application of machine learning techniques for driving errors analysis: systematic literature reviewInternational Journal of Crashworthiness10.1080/13588265.2023.230114629:5(785-793)Online publication date: 7-Jan-2024
      • (2023)A Survey on Measuring Cognitive Workload in Human-Computer InteractionACM Computing Surveys10.1145/358227255:13s(1-39)Online publication date: 13-Jul-2023
      • (2022)Towards Implicit Interaction in Highly Automated Vehicles - A Systematic Literature ReviewProceedings of the ACM on Human-Computer Interaction10.1145/35467266:MHCI(1-21)Online publication date: 20-Sep-2022
      • (2021)Multimodal Capture of Patient Behaviour for Improved Detection of Early Dementia: Clinical Feasibility and Preliminary ResultsFrontiers in Computer Science10.3389/fcomp.2021.6426333Online publication date: 19-Apr-2021
      • (2020)Towards Mindless Stress Regulation in Advanced Driver Assistance Systems: A Systematic ReviewFrontiers in Psychology10.3389/fpsyg.2020.60912411Online publication date: 23-Dec-2020
      • (2019)Text ComprehensionProceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3342197.3344547(342-354)Online publication date: 21-Sep-2019
      • (2018)Machine Learning Approaches to Automatic Stress Detection: A Review2018 IEEE/ACS 15th International Conference on Computer Systems and Applications (AICCSA)10.1109/AICCSA.2018.8612825(1-6)Online publication date: Oct-2018
      • (2014)Evaluation of Driver Stress while Transiting Road TunnelsAdjunct Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/2667239.2667269(1-6)Online publication date: 17-Sep-2014
      • (2014)Emotion recognition from thermal infrared images using deep Boltzmann machineFrontiers of Computer Science10.1007/s11704-014-3295-38:4(609-618)Online publication date: 14-Mar-2014
      • (2013)A data set of real world driving to assess driver workloadProceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/2516540.2516561(150-157)Online publication date: 28-Oct-2013
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media