Skip to main content

Advertisement

SpringerLink
Log in

Respiratory signal and human stress: non-contact detection of stress with a low-cost depth sensing camera

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Psychological stress may cause various health problems. To prevent the potential chronic illness that long-term psychological stress could cause, it is important to detect and monitor the psychological stress at its initial stage. In this paper, we present a framework for remotely detecting and classifying human stress by using a KINECT sensor that is portable and affordable enough for ordinary users in everyday life. Unlike most of emotion recognition tasks in which respiratory signals (RSPS) are usually used only as an aiding analysis, the whole task proposed is based on RSPS. Thus, the main contribution of this paper is that not only the non-contact devices is used to identify human stress, but also the relationship between RSPS and stress recognition is analyzed in detail. Experimental results on 84 volunteers show that the recognition accuracy for recognizing psychological stress, physical stress, and relaxing state are 93.90%, 93.40%, and 89.05% respectively. These results suggest that the proposed framework is effective for monitoring human stress, and RSPS could be used for stress recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Hong K, Yuen P, Chen T, Tsitiridis A, Kam F, Richardson M, James D, Oxford W, Piper J, Thomas F, Lightman S (2009) Detection and classification of stress using thermal imaging technique. Proceedings SPIE 7486:01

    Google Scholar 

  2. Peng M, Wang C, Chen T, Liu G, Fu X (2017) Dual temporal scale convolutional neural network for micro-expression recognition. Front Psychol 8:1745

    Article  Google Scholar 

  3. Wu Z, Chen T, Chen Y et al (2017) NIRExpNet: three-stream 3D convolutional neural network for near infrared facial expression recognition. Appl Sci 7(11):1184

    Article  Google Scholar 

  4. Barros P, Parisi GI, Weber C et al (2017) Emotion-modulated attention improves expression recognition: a deep learning model[J]. Neurocomputing 253:104–114

    Article  Google Scholar 

  5. Tzirakis P, Trigeorgis G, Nicolaou MA et al (2017) End-to-end multimodal emotion recognition using deep neural networks[J]. IEEE J Sel Topics Signal Process 11(8):1301–1309

    Article  Google Scholar 

  6. Ng H W, Nguyen V D, Vonikakis V, et al (2015) Deep learning for emotion recognition on small datasets using transfer learning[C]/Proceedings of the 2015 ACM on international conference on multimodal interaction. ACM, 2015: 443–449.

  7. Rosalind Picard’s Speech in TED 2011. https://www.youtube.com/watch?v=ujxriwApPP4&t=354s. Accessed 22 Feb 2020

  8. J. Healey and R. Picard (2000) “SmartCar: Detecting driver stress,” in Proc. 15th Int. Conf. Pattern Recognit, 2000.

  9. Kim KH, Bang SW, Kim SR (2004) Emotion recognition system using short-term monitoring of physiological signals. Med Biol Eng Comput 42:419–427

    Article  Google Scholar 

  10. I. Pavlidis, J. Levine and P. Baukol (2000) “Thermal imaging for anxiety detection,” in Proc. IEEE Workshop Comput. Vis. Beyond Vis. Spectrum:Methods Appl., 2000.

  11. Chen T, Yuen P, Richardson M, Liu G, She Z (2014) Detection of psychological stress using a hyperspectral imaging technique. IEEE Trans Affect Comput 5(4):391–405

    Article  Google Scholar 

  12. T. Chen, P. Yuen, K. Hong, A. Tsitiridis, F. Kam, J. Jackman, D. James, M. Richardson, W. Oxford, J. Piper, F. Thomas and S. Lightman (2009) “Remote sensing of stress using electro-optics imaging technique,” Proc. SPIE 7486, Optics and Photonics for Counterterrorism and Crime Fighting V, 748606, 9 2009.

  13. D. McDuff, S. Gontarek and R. Picard, “Remote measurement of cognitive stress via heart rate variability,” in 36th Annual International Conference of the IEEE on Engineering in Medicine and Biology Society, 2014.

  14. Pavlidis I, Dowdall J, Sun N, Puri C, Fei J, Garbey M (2007) Interacting with human physiology. Comput Vis Image Underst 108:150–170

    Article  Google Scholar 

  15. Shastri D, Papadakis M, Tsiamyrtzis P, Bass B, Pavlidis I (2012) Perinasal imaging of physiological stress and its affective potential. IEEE Trans Affect Comput 3(3):366–378

    Article  Google Scholar 

  16. Fernández JRM, Anishchenko L (2018) Mental stress detection using bioradar respiratory signals[J]. Biomed Signal Process Control 43:244–249

    Article  Google Scholar 

  17. Lundber U, Forsman M, Zachau G, Eklof M, Palmer G, Melin B, Kadefors R (2002) Effects of experimentally induced mental and physical stress on motor unit recruitment in the trapezius muscle. Work Stress 16(2):166–178

    Article  Google Scholar 

  18. Hong K, Liu G, Chen W et al (2018) Classification of the emotional stress and physical stress using signal magnification and canonical correlation analysis[J]. Pattern Recogn 77:140–149

    Article  Google Scholar 

  19. Shan Y et al. (2018) “Remote Detection and Classification of Human Stress Using a Depth Sensing Technique.” 2018 First Asian Conference on Affective Computing and Intelligent Interaction (ACII Asia). IEEE, 2018.

  20. Boiten FA, Frijda NH, Wientjes CJ (1994) Emotions and respiratory patterns: review and critical analysis. Int J Psychophysiol 17(2):103–128

    Article  Google Scholar 

  21. Del Negro C A, Funk G D, Feldman J L. Breathing matters[J]. Nat Rev Neurosci, 2018.

  22. Colasanti A, Salamon E, Schruers K et al (2008) Carbon dioxide-induced emotion and respiratory symptoms in healthy volunteers[J]. Neuropsychopharmacology 33(13):3103

    Article  Google Scholar 

  23. Mocanu E, Mohr C, Pouyan N et al (2018) Reasons, years and frequency of yoga practice: effect on emotion response reactivity[J]. Front Human Neurosci 12:264

    Article  Google Scholar 

  24. Lim R, Zavou MJ, Milton PL et al (2014) Measuring respiratory function in mice using unrestrained whole-body plethysmography[J]. JoVE 2014:90

    Google Scholar 

  25. Garde A, Giraldo B F, Sörnmo L, et al. (2011) Analysis of the respiratory flow cycle morphology in chronic heart failure patients applying principal components analysis[C]//Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE. IEEE, 2011: 1725–1728.

  26. Clare M, Hopper K (2005) Mechanical ventilation: ventilator settings, patient management, and nursing care[J]. Compend Contin Educ Pract Vet 27(4):256–268

    Google Scholar 

  27. Cohen HD, Goodenough DR, Witkin HA et al (1975) The effects of stress on components of the respiration cycle[J]. Psychophysiology 12(4):377–380

    Article  Google Scholar 

  28. Van De Bruaene A, Claessen G, La Gerche A et al (2015) Effect of respiration on cardiac filling at rest and during exercise in Fontan patients: a clinical and computational modeling study[J]. IJC Heart Vasculature 9:100–108

    Article  Google Scholar 

  29. Bloch S, Lemeignan M, Aguilera-T N (1991) Specific respiratory patterns distinguish among human basic emotions. Int J Psychophysiol 11(2):141–154

    Article  Google Scholar 

  30. Philippot P, Chapelle G, Blairy S (2002) Respiratory feedback in the generation of emotion. Cogn Emot 16(5):605–627

    Article  Google Scholar 

  31. Suess WM, Alexander AB, Smith DD et al (1980) The effects of psychological stress on respiration: a preliminary study of anxiety and hyperventilation[J]. Psychophysiology 17(6):535–540

    Article  Google Scholar 

  32. Gjoreski M, Gjoreski H, Luštrek M, et al. Continuous stress detection using a wrist device: in laboratory and real life[C]//Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. ACM, 2016: 1185–1193.

  33. F. Bousefsaf, C. Maaoui and A. Pruski, “Remote assessment of the heart rate variability to detect mental stress[C]//,” in IEEE 7th International Conference on Pervasive Computing Technologies for Healthcare, 2013.

  34. McDuff D, Hernandez PR (2016) “Contact-free measurement of cognitive stress during computer tasks with a digital camera,” in Computer and Human Interaction Conference (CHI). California, San Jose

    Google Scholar 

  35. Shao D, Yang Y, Liu C, Liu C, Tsow F, Yu H, Tao N (2014) Noncontact monitoring breathing pattern, exhalation flow rate and pulse transit time. IEEE Trans Biomed Eng 61(11):2760–2767

    Article  Google Scholar 

  36. Zhao F, Li M, Qian Y, Tsien J (2013) Remote measurements of heart and respiration rates for telemedicine. PLoS ONE 8(10):e71384

    Article  Google Scholar 

  37. Al-Khalidi F, Saatchi R, Elphick H, Burke D (2011) An evaluation of thermal imaging based respiration rate monitoring in children. Am J Eng Appl Sci 4(4):586–597

    Article  Google Scholar 

  38. H. Elphick, A. Alkali, R. Kingshott, D. Burke and R. Saatchi (2015) “Thermal imaging method for measurement of respiratory rate,” European Respiratory Journal, 46(59):PA1260, 2015.

  39. B. Xu, L. K. Mestha and G. Pennington (2014) “Monitoring respiration with a thermal imaging system”. US Patent US8790269B2, 2014.

  40. Uenoyama M, Matsui T, Yamada K, Suzuki S, Takase B, Suzuki S, Ishihara M, Kawakami M (2006) Non-contact respiratory monitoring system using a ceiling-attached microwave antenna. Med Biol Eng Compu 44(9):835–840

    Article  Google Scholar 

  41. Lee YS, Pathirana PN, Steinfort CL, Caelli T (2014) Monitoring and analysis of respiratory patterns using microwave doppler radar. IEEE J Transl Eng Health Med 2:1–12

    Article  Google Scholar 

  42. Lee YS, Pathirana PN, Evans RJ, Steinfort CL (2015) Noncontact detection and analysis of respiratory function using microwave Doppler radar. J Sensors 2015:548136

    Google Scholar 

  43. Gu C, Li C (2015) Assessment of human respiration patterns via noncontact sensing using doppler multi-radar system[J]. Sensors 15(3):6383–6398

    Article  Google Scholar 

  44. N. Bernacchia, L. Scalise, L. Casacanditella, I. Ercoli, P. Marchionni and E. P. Tomasini (2014) “Non contact measurement of heart and respiration rates based on Kinect,” in 2014 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Lisboa, 2014.

  45. F. Tahavori, M. Alnowami and K. Wells (2014) “Marker-less respiratory motion modeling using the Microsoft Kinect for Windows,” in SPIE Medical Imaging. International Society for Optics and Photonics, 2014.

  46. F. Tahavori, E. Adams, M. Dabbs, L. Aldridge, N. Liversidge, E. Donovan, T. Jordan, P. Evans and K. Wells (2015) “Combining marker-less patient setup and respiratory motion monitoring using low cost 3D camera technology,” in Proc. SPIE 9415, Medical Imaging 2015, Orlando, 2015.

  47. H. Aoki, M. Miyazaki, H. Nakamura, R. Furukawa, R. Sagawa and H. Kawasaki (2012) “Non-contact respiration measurement using structured light 3-d sensor,” in 2012 Proceedings of SICE Annual Conference (SICE), Akita, 2012.

  48. Kuo Y-M, Lee J-S, Chung P-C (2010) A visual context-awareness-based sleeping-respiration measurement system. IEEE Trans Inf Technol Biomed 14(2):255–265

    Article  Google Scholar 

  49. Cho Y, Bianchi-Berthouze N, Julier S J (2017) DeepBreath: Deep learning of breathing patterns for automatic stress recognition using low-cost thermal imaging in unconstrained settings[C]//Affective Computing and Intelligent Interaction (ACII), 2017 Seventh International Conference on. IEEE, 2017: 456–463.

  50. Xia J, Siochi RA (2012) A real-time respiratory motion monitoring system using KINECT: proof of concept. Med Phys 39(5):2682–2685

    Article  Google Scholar 

  51. Tulen JHM, Moleman P, Van Steenis HG et al (1989) Characterization of stress reactions to the Stroop Color Word Test[J]. Pharmacol Biochem Behav 32(1):9–15

    Article  Google Scholar 

  52. Hjemdahl P, Freyschuss U, Juhlin-Dannfelt A et al (1984) Differentiated sympathetic activation during mental stress evoked by the Stroop test[J]. Acta Physiol Scand Suppl 527:25–29

    Google Scholar 

  53. Golden C J, Freshwater S M (1978) Stroop color and word test[J]. 1978.

  54. Aigrain J (2016) Multimodal detection of stress: evaluation of the impact of several assessment strategies[D]. Paris 6, 2016.

  55. Breiman L (2001) Random forests. Mach Learn 45:5–32

    Article  MATH  Google Scholar 

  56. Jebelli H, Hwang S, Lee SangHyun (2018) EEG-based workers’ stress recognition at construction sites. Autom Constr 93:315–324

    Article  Google Scholar 

  57. Zangróniz R, Martínez-Rodrigo A, Pastor J et al (2017) Electrodermal activity sensor for classification of calm/distress condition[J]. Sensors 17(10):2324

    Article  Google Scholar 

  58. Sriramprakash S, Prasanna VD, Murthy OVR (2017) Stress detection in working people[J]. Proc Comput Sci 115:359–366

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially funded by the National Natural Science Foundation of China (Grant No. 61301297, 61502398).

Author information

Authors and Affiliations

  1. Chongqing Key Laboratory of Non-Linear Circuit and Intelligent Information Processing, Southwest University, Chongqing, 400715, China

    Yuhao Shan, Shigang Li & Tong Chen

  2. School of Information Science, Hiroshima City University, Hiroshima, 731-3194, Japan

    Yuhao Shan & Shigang Li

  3. Chongqing Key Laboratory of Artificial Intelligence and Service Robot Control Technology, Chongqing, 400715, China

    Tong Chen

Authors
  1. Yuhao Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shigang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shigang Li or Tong Chen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 3794 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shan, Y., Li, S. & Chen, T. Respiratory signal and human stress: non-contact detection of stress with a low-cost depth sensing camera. Int. J. Mach. Learn. & Cyber. 11, 1825–1837 (2020). https://doi.org/10.1007/s13042-020-01074-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-020-01074-x

Keywords

Search

Navigation