Skip to main content

Advertisement

SpringerLink
Log in

Real-time stress assessment using thermal imaging

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Thermal imaging is one of the most promising methods of probing the psychological status of human beings because of its non-invasiveness. This study develops a new method of assessing the stress status in real time. The differential energy between philtrum (located in the maxillary area) and forehead (DEFP) algorithm is developed to amplify and extract stress-induced thermal imprints. The algorithm is then validated against the clinical standard in a controlled lab experiment in which neurophysiologic responses are invoked from subjects via Trier Social Stress Test. The correlation between the extracted thermal imprints and established stress markers (heart beat rate and cortisol level) is significant. Experimental result demonstrates that DEFP has the capacity to classify stress and baseline status. An accuracy rate of over 90 % suggests the feasibility of the real-time assessment of stress, disregarding personal factors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Natrual Tech: The Adrenal Stress Profile. http://www.natratech.com/Products/saliva_test.htm. Accessed 7 March 2012

  2. Read, G.F.: Immunoassays of steroids in saliva. Steroid Biochem. 22(33), 437–438 (1985)

    Google Scholar 

  3. Sauro, M., Jorgensen, R., et al.: Sociotropic cognition moderates anxiety-induced cardiovascular responsiveness in college women. Behav. Med. 24(5), 423–439 (2001)

    Article  Google Scholar 

  4. Condren, R., et al.: HPA axis response to a psychological stressor in generalised social phobia. Psychoneuroendocrinology 27(6), 693–703 (2002)

    Article  Google Scholar 

  5. Gunes, H., Schuller, B., Pantic, M., Cowie, R.: Emotion representation, analysis and synthesis in continuous space: a survey. In: Proceedings of the 1st International Workshop on Emotion Synthesis, Representation, and Analysis in Continuous Space, pp. 827-834. IEEE, Santa Barbara (2011)

  6. Berry, J.W., et al.: Forgivingness, relationship quality, anxiety while imagining relationship events, and physical and mental health. Couns. Psychol. 48, 447–455 (2011)

    Article  Google Scholar 

  7. Hellhammer, D.H., Wust, S., Kudielka, B.M.: Salivary cortisol as a biomarker in stress research. Psychoneuroendocrinology 34, 163–171 (2009)

    Article  Google Scholar 

  8. Williams, H., Read, G., et al.: Effect of inhaled beclomethasone dipropionate on saliva cortisol concentrations. Arch. Dis. Child 59(6), 553–556 (1984)

    Article  Google Scholar 

  9. Wing, R.R.: Psychologic anxiety and blood glucose levels in nondiabetic subjects. Psychosom. Med. 47(6), 102–107 (1985)

    Article  Google Scholar 

  10. Federenko, I.S., Nagamine, M., Hellhammer, D.H., Wadhwa, P.D., Wüst, S.: The heritability of hypothalamus pituitary adrenal axis responses to psychosocial stress is context dependent. Clin. Endocrinol. Metab. 89(12), 127–136 (2004)

    Article  Google Scholar 

  11. Kudielka, B.M., et al.: Sex differences in HAP axis response to anxiety: a review. Biol. Psychol. 69, 113–132 (2004)

    Article  Google Scholar 

  12. Kudielka, B.M., Hellhammer, D.H.: Acute HPA axis responses, heart rate and mood changes to psychosocial anxiety (TSST) in humans at different times of day. Psychoneuroendocrinology 29, 983–992 (2004)

    Article  Google Scholar 

  13. Ishii, H., Niioka, T., Izumi, H.: Circulating adrenaline released by sympatho adrenal activation elicits acute vasodilatation in the rat masseter muscle. Arch. Oral Biol. 54(5), 486–494 (2009)

    Article  Google Scholar 

  14. Healey, J.A., Picard, R.W.: Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans. Intell. Transp. Syst. 6(2), 156–166 (2005)

    Article  Google Scholar 

  15. Barreto, A., Zhai, J., Adjouadi, M.: Non-intrusive physiological monitoring for automated stress detection in human–computer interaction. In: Lecture Notes in Computer Science. Human–Computer Interaction, pp. 29–38 (2007)

  16. Shi, Y., Nguyen, M.H., Blitz, P., French, B., Fisk, S., De la Torre, F., Smailagic, A., Siewiorek, D.P., Kumar, S.: Personalized stress detection from physiological measurements. In: International Symposium on Quality of Life Technology, pp. 28–29 (2010)

  17. Qiang, W., et al.: Fractal dimension based neurofeedback in serious games. Vis. Comput. 27(4), 299–309 (2011)

    Article  MathSciNet  Google Scholar 

  18. Stephanos, I., Vittorio, G., Arcangelo, M.: Thermal infrared imaging in psychophysiology: potentialities and limits. Psychophysiology 51, 951–963 (2014)

    Article  Google Scholar 

  19. Pavlidis, I., et al.: Human behavior: seeing through the face of deception. Nature 415(6867), 35–36 (2002)

    Article  Google Scholar 

  20. Pavlidis, I. et al.: Continuous physiological monitoring. In: Proceedings of the 25th Annual International Conference of the IEEE EMBS Cancun, Mexico, pp. 17–21 (2003)

  21. Pavlidis, I., Levine, J., Baukol, P.: Thermal image analysis for anxiety detection. In: Proceedings of the 2001 IEEE International Conference on Image Processing, vol. 2, pp. 315–318 (2001)

  22. Pavlidis, I., Dowdall, J., et al.: Interacting with human physiology. Comput. Vis. Image Underst. 108, 150–170 (2007)

    Article  Google Scholar 

  23. Pavlidis, I., et al.: Fast by nature—how stress patterns define human experience and performance in dexterous tasks. Sci. Rep. 2, 1–9 (2012)

    Article  Google Scholar 

  24. Calvin, K.L., et al.: Development of a facial skin temperature-based methodology for non-intrusive mental workload measurement. Occup. Ergon. 7, 83–94 (2007)

    Google Scholar 

  25. Drummond, P.: The effect of adrenergic blockade on blushing and facial flushing. Psychophysiology 34(2), 163–168 (1997)

    Article  Google Scholar 

  26. Drummond, P.: The effect of anger and pleasure on facial blood flow. Austr. J. Psychol. 46(2), 95–99 (1994)

    Article  Google Scholar 

  27. Drummond, P., Lance, J.: Facial flushing and sweating mediated by the sympathetic nervous system. Brain 110(3), 793–803 (1987)

    Article  Google Scholar 

  28. Pavlidis, I., Tsiamyrtzis, P., Shastri, D., Wesley, A., Zhou, Y., et al.: Fast by nature—how stress patterns define human experience and performance in dexterous tasks. Sci. Rep. 2, 305–306 (2012)

    Article  Google Scholar 

  29. Ebisch, S.J., Aureli, T., Bafunno, D., Cardone, D., Romani, G.L., et al.: Mother and child in synchrony: thermal facial imprints of autonomic contagion. Biol. Psychol. 89, 123–129 (2012)

    Article  Google Scholar 

  30. Vatner, S.F.: Effects of hemorrhage on regional blood flow distribution in dogs and primates. Clin. Invest. 54, 225–235 (1974)

    Article  Google Scholar 

  31. Calvin, K.L., Duffy, V.G.: Development of a facial skin temperature-based methodology for non-intrusive mental workload measurement. Occup. Ergon. 7, 83–94 (2007)

    Google Scholar 

  32. Anbar, M.: Assessment of physiologic and pathologic radiative heat dissipation using dynamic infrared imaging. Ann. N. Y. Acad. Sci. 972, 111–118 (2002)

    Article  Google Scholar 

  33. Jarlier, S., Grandjean, D., Delplanque, S., N’Diaye, K., Cayeux, I., Velazco, M.I., Sander, D., Vuilleumier, P., Scherer, K.R.: Thermal analysis of facial muscles contractions. IEEE Trans. Affect. Comput. 2(1), 2–9 (2011)

    Article  Google Scholar 

  34. Veronika, E., et al.: Exploring the use of thermal infrared imaging in human stress research. PLoS One 9(3), 125–136 (2014)

    Google Scholar 

  35. Vijesh, S., et al.: Imaging facial signs of neurophysiological responses. IEEE Trans. Biomed. Eng. 56, 477–484 (2009)

    Article  Google Scholar 

  36. Nozawa, A., Tacano, M.: Correlation analysis on alpha attenuation and nasal skin temperature. Stat. Mech. Theory Exp. Impact Factor Inf. 1007, 1–10 (2009)

    Google Scholar 

  37. Levine, J., Pavlidis, I., et al.: Description and clinical studies of a device for the instantaneous detection of office-place anxiety. Work 34(3), 359–364 (2009)

    Google Scholar 

  38. Levine, J.A., Pavlidis, I., Cooper, M.: The face of fear. Lancet 357(9270), 1757–1757 (2001)

    Article  Google Scholar 

  39. Nhan, B., Chau, T.: Classifying affective states using thermal infrared imaging of the human face. IEEE Trans. Biomed. Eng. 57(4), 887–979 (2010)

    Article  Google Scholar 

  40. Giacinto, A., et al.: Thermal signature of fear conditioning in mild posttraumatic stress disorder. Neuroscience 266, 216–223 (2014)

    Article  Google Scholar 

  41. Rashid, M., et al.: Human emotion recognition from videos using spatio-temporal and audio features. Vis. Comput. 29(12), 1269–1275 (2012)

  42. Ioannou, S., Ebisch, S.J., Aureli, T., Bafunno, D., Ioannides, H.A., Cardone, D., Merla, A.: The autonomic signature of guilt in children: a thermal infrared imaging study. PloS One 8, 1–11 (2013)

    Article  Google Scholar 

  43. Kozel, F.A., Johnson, K.A.: Detecting deception using functional agnetic resonance imaging. Biol. Psychiatry 58, 605–613 (2005)

    Article  Google Scholar 

  44. Bles, M., Haynes, J.D.: Detecting concealed information using brain imaging technology. Neurocase Neural Basis Cognit. 14(1), 82–92 (2008)

    Google Scholar 

  45. Bhatt, S., Mbwana, J., Adeyemo, A., et al.: Lying about facial recognition: an fMRI study. Brain Cognit. 69, 382–390 (2009)

    Article  Google Scholar 

  46. Kozel, F.A., Johnson, K.A., et al.: Functional MRI detection of deception after committing a mock sabotage crime. Forensic Sci. 54(1), 220–231 (2009)

    Article  Google Scholar 

  47. Madrigal, A.: Wired Science. 16 March 2009. http://www.wired.com/wiredscience/2009/03/noliemri/. Accessed 5 March 2012

  48. Konstantinos, C., et al.: A framework for implicit human-centered image tagging inspired by attributed affect. Vis. Comput. 30(1), 1093–1106 (2014)

    Google Scholar 

  49. Jarlier, S., et al.: Thermal analysis of facial muscles contractions. IEEE Trans. Affect. Comput. 2(1), 2–9 (2011)

    Article  Google Scholar 

  50. Stewart, M., Webster, J., et al.: Non-invasive measurement of anxiety in dairy cows using infrared thermography. Physiol. Behav. 92(3), 520–525 (2007)

    Article  Google Scholar 

  51. Nakayama, K., Goto, S., et al.: Decrease in nasal temperature of rhesus monkeys (Macacamulatta) in negative emotional state. Physiol. Behav. 84(5), 783–790 (2005)

    Article  Google Scholar 

  52. Zawa, A., et al.: Dynamic analysis of dorsal thermal images. Artif. Life Robot. 16, 147–151 (2011)

    Article  Google Scholar 

  53. Vianna, D., Carrive, P.: Changes in cutaneous and body temperature during and after conditioned fear to context in the rat. Eur. J. Neurosci. 21, 2505–2510 (2005)

    Article  Google Scholar 

  54. Kuraoka, K., Nakamura, K.: The use of nasal skin temperature measurements in studying emotion in macaque monkeys. Physiol. Behav. 102, 347–355 (2011)

    Article  Google Scholar 

  55. Dvijesh, S., Ioannis, P.: Perinasal imaging of physiological stress and its affective potential. IEEE Trans. Affect. Comput. 3(3), 366–378 (2012)

    Article  Google Scholar 

  56. Gault, T., et al.: Extraction of the superficial facial vasculature, vital signs waveforms and rates using thermal imaging. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–8 (2010)

  57. Petkie, D.T., et al.: Millimeter-wave radarystems for biometric applications. Proc. SPIE Eur. Secur. Def. Conf. 7, 1–7 (2009)

    Google Scholar 

  58. Petkie, D.T., Bryan, E.: Millimeter wave radar for remote measurement of vital signs. IEEE Radar Conf. 1, 1–3 (2009)

    Google Scholar 

  59. Fei, J., Zhu, Z., Pavlidis, I.: Imaging breathing rate in the CO\(_2\) absorption band. In: 27th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 700–705 (2005)

  60. Fei, J., Pavlidis, I.: Thermistor at a distance: unobtrusive measurement of breathing. IEEE Trans. Biomed. Eng. 57(4), 988–998 (2010)

    Article  Google Scholar 

  61. Garbey, M., Pavlidis, I.: Contact-free measurement of cardiac pulse based on the analysis of thermal imagery. IEEE Trans. Biomed. Eng. 54(8), 1418–1426 (2007)

    Article  Google Scholar 

  62. Chekmenev, S., et al.: Multi-Resolution Approach for Noncontact Measurements of Arterial Pulse Using Thermal Imaging. Augmented Vision Perception in Infrared. Springer, London (2009)

    Google Scholar 

  63. Hao-Yu, W., Michael, R., et al.: Eulerian video magnification for revealing subtle changes in the world. ACM Trans. 31(4), 1–8 (2012)

    Article  Google Scholar 

  64. Nakasone, A., Prendinger, H., Ishizuka, M.: Emotion recognition from electromyography and skin conductance. In: Proceedings of the International Workshop on Biosignal Interpretation, pp. 219–222 (2005)

  65. Wollmer, M., et al.: Abandoning emotion classes—towards continuouse motion recognition with modelling of long-range dependencies. In: Proceedings of the Interspeech, pp. 597–600 (2008)

  66. Nicolaou, M.A., et al.: Audio-visual classification and fusion of spontaneous affective data in likelihood space. In: Proceedings of the International Conference on Pattern Recognition, pp. 3695–3699 (2010)

  67. Petridis, S., et al.: Static vs. dynamic modeling of human nonverbal behavior from multiple cues and modalities. In: Proceedings of the ACM ICMI, pp. 23–30 (2009)

  68. Hahn, A., et al.: Hot or not? Thermal reactions to social contact. Biol. Lett. (Physiol.) 8, 1–4 (2012)

    Article  Google Scholar 

  69. Chourmouzios, T., Maria, P.: On the choice of the parameters for anisotropic diffusion in image processing. Pattern Recognit. 46(5), 1369–1381 (2013)

    Article  Google Scholar 

  70. Daubechies, : Ten Lectures on Wavelets. SIAM, Philadelphia (1992)

    Book  MATH  Google Scholar 

  71. Hong, S., Zhou, Z., Zio, E., Hong, K.: Condition assessment for the performance degradation of bearing based on a combinatorial feature extraction method. Digit Signal Process. 27, 159–166 (2014)

    Article  Google Scholar 

  72. Ebisch, S.J., et al.: Mother and child in synchrony: thermal facial imprints of autonomic contagion. Biol. Psychol. 89, 123–129 (2012)

    Article  Google Scholar 

  73. Schommer, N.C., et al.: Dissociation between reactivity of the hypothalamus–pituitary–adrenal axis and the sympathetic adrenal-medullary system to repeated psychosocial stress. Psychosom. Med. 65, 450–460 (2003)

    Article  Google Scholar 

  74. Dowdall, J., et al.: Coalitional tracking. Comput. Vis. Image Underst. 106(2), 205–219 (2007)

    Article  Google Scholar 

  75. Zhou, Y., Tsiamyrtzis, P., Pavlidis, I., et al.: Spatiotemporal smoothing as a basis for facial tissue tracking in thermal imaging. IEEE Trans. Biomed. Eng. 60(5), 1280–1289 (2013)

    Article  Google Scholar 

Download references

Acknowledgments

The authors are highly thankful for the financial support of National Natural Science Foundation of China (61304111), Beijing Natural Science Foundation (4153059), National Program on Key Basic Research Program of China under grant 2014CB744904, and Fundamental Research Funds for the Central Universities under grant No. YWF-14-KKX-001, China.

Author information

Authors and Affiliations

  1. Optoelectronics and Communication Engineering Key Laboratory, Jiangxi Science and Technology Normal University, Nanchang, Jiangxi, China

    Kan Hong

  2. Science and Technology on Reliability and Environmental Engineering Laboratory, School of Reliability and System Engineering, Beihang University, No. 37, Xue Yuan Road, Beijing, 100191, China

    Sheng Hong

Authors
  1. Kan Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sheng Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sheng Hong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hong, K., Hong, S. Real-time stress assessment using thermal imaging. Vis Comput 32, 1369–1377 (2016). https://doi.org/10.1007/s00371-015-1164-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-015-1164-1

Keywords

Search

Navigation