Skip to main content

Advertisement

SpringerLink
Log in

Discriminating affective state intensity using physiological responses

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper we explore if physiological signals obtained from a person through wearable sensors permit the correct interpretation of human affective states. A crucial aspect within this field of research is the capability of inducing real-life emotions in laboratory environments. To this end we designed a very strict and regulated experimental protocol. We consider two affective states: a relaxation state and a stressful one. To induce these two states we adopt audio tracks of natural or day-life sounds and math calculations. Moreover, to study different intensities of effectively induced relaxation, we consider two different audio players: a traditional pair of headphones and the Spherison Sound6D pillowⒸ, a special device that provides a complete spherical immersion in what users are listening to. We consider as physiological signals the Galvanic Skin Response (GSR) and the Photoplethysmography (PPG), as they are sensitive measures for emotional arousal. After an extensive analysis on our experimental data, we demonstrate that GSR and PPG signals can successfully distinguish relaxation and stressful states. Moreover, the same physiological signals can discriminate affective state intensity, especially when relaxation is induced adopting the Sound6D technology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Abadi MK, Subramanian R, Kia SM, Avesani P, Patras I, Sebe N (2015) DECAF: MEG-Based multimodal database for decoding affective physiological responses. IEEE Trans Affect Comput 6(3):209–222

    Article  Google Scholar 

  2. Alvarsson JJ, Wiens S, Nilsson ME (2010) Stress recovery during exposure to nature sound and environmental noise. Inte J Environ Res Public Health 7(3):1036–1046

    Article  Google Scholar 

  3. Bandini S, Gasparini F (2020) Towards affective walkability for healthy ageing in the future of the cities. In: Bandini S, Cortellessa G, Palumbo F (eds) Proceedings of the Fifth Italian Workshop on Artificial Intelligence for Ambient Assisted Living, vol. 2559, pp. 1–9. CEUR-WS

  4. Bandini S, Gasparini F, Giltri M (2019) Personalized music experience for the wellbeing of elderly people. In: El Yacoubi S, Bagnoli F, Pacini G (eds) Lecture notes in computer science, vol 11938. Springer International Publishing, Cham, pp 335–340

  5. Bradley MM, Lang PJ (1994) Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Therapy Exp Psychiat 25(1):49–59

    Article  Google Scholar 

  6. Bradley MM, Lang PJ (2007) The International Affective Digitized Sounds(; IADS-2): Affective ratings of sounds and instruction manual. University of Florida, Gainesville, FL, Tech. Rep B-3

  7. Burns A, Doheny EP, Greene BR, Foran T, Leahy D, O’Donovan K, McGrath MJ (2010) SHIMMER™: an extensible platform for physiological signal capture. In: 2010 Annual international conference of the IEEE engineering in medicine and biology. IEEE, Los Alamitos, pp 3759–3762

  8. Can YS, Arnrich B, Ersoy C (2019) Stress detection in daily life scenarios using smart phones and wearable sensors: A survey. Journal of biomedical informatics p 103139

  9. Chen LL, Zhao Y, Ye PF, Zhang J, Zou JZ (2017) Detecting driving stress in physiological signals based on multimodal feature analysis and kernel classifiers. Expert Syst Appl 85:279–291

    Article  Google Scholar 

  10. Chiu MC, Ko LW (2017) Develop a personalized intelligent music selection system based on heart rate variability and machine learning. Multimedia Tools and Applications 76(14):15607–15639

    Article  Google Scholar 

  11. Chung JW, Vercoe GS (2006) The affective remixer: Personalized music arranging. In: CHI’06 Extended abstracts on human factors in computing systems, pp. 393–398. ACM

  12. Cliff N (1993) Dominance statistics: Ordinal analyses to answer ordinal questions. Psychol Bull 114(3):494

    Article  Google Scholar 

  13. Clynes M (1977) Sentics: The touch of emotions Anchor Press

  14. Dan-Glauser ES, Scherer KR (2011) The Geneva affective picture database (GAPED): a new 730-picture database focusing on valence and normative significance. Behav Res Methods 43(2):468

    Article  Google Scholar 

  15. Dey N, Ashour AS, Shi F, Fong SJ, Tavares JMR (2018) Medical cyber-physical systems: a survey. J Med Syst 42(4):74

    Article  Google Scholar 

  16. Ekman P (1992) An argument for basic emotions. Cognit Emotion 6(3-4):169–200

    Article  Google Scholar 

  17. Girardi D, Lanubile F, Novielli N (2017) Emotion detection using noninvasive low cost sensors. In: 2017 Seventh international conference on affective computing and intelligent interaction (ACII). IEEE, Los Alamitos, pp 125–130

  18. Guess H (2017) Alzheimer’s disease and the impact of music therapy a systematic literature review

  19. Hakimi N, Setarehdan SK (2018) Stress assessment by means of heart rate derived from functional near-infrared spectroscopy. J Biomed Opt 23(11):115001

    Google Scholar 

  20. Healey JA, Picard RW (2005) Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans Int Trans Syst 6(2):156–166

    Article  Google Scholar 

  21. Hernandez J, Morris RR, Picard RW (2011) Call center stress recognition with person-specific models. In: International conference on affective computing and intelligent interaction. Springer, New York, pp 125–134

  22. Hernando-Gallego F, Artés-rodríguez A (2015) Individual performance calibration using physiological stress signals. arXiv:1507.03482

  23. Hönig F., Batliner A, Nöth E. (2007) Real-time recognition of the affective user state with physiological signals. Proceedings of the Doctoral Consortium. Affect Comput Int Interact 28

  24. Hu WL, Meyer JJ, Wang Z, Reid T, Adams DE, Prabnakar S, Chaturvedi AR (2015) Dynamic data driven approach for modeling human error. Procedia Comput Sci 51:1643–1654

    Article  Google Scholar 

  25. Janssen JH, Van Den Broek EL, Westerink JH (2012) Tune in to your emotions: a robust personalized affective music player. User Model User-Adap Inter 22 (3):255–279

    Article  Google Scholar 

  26. Koelstra S, Muhl C, Soleymani M, Lee JS, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2011) Deap: a database for emotion analysis; using physiological signals. IEEE trans Affect Comput 3(1):18–31

    Article  Google Scholar 

  27. Koldijk S, Sappelli M, Verberne S, Neerincx MA, Kraaij W (2014) The swell knowledge work dataset for stress and user modeling research. In: Proceedings of the 16th international conference on multimodal interaction, pp 291–298

  28. Lang PJ, Bradley MM, Cuthbert BN (1997) International affective picture system (IAPS): Technical manual and affective ratings. NIMH Center Study Emot Atten 1:39–58

    Google Scholar 

  29. Marchewka A, ŻUrawski Ł, Jednoróg K, Grabowska A (2014) The Nencki Affective Picture System (NAPS,): Introduction to a novel, standardized, wide-range, high-quality, realistic picture database. Behav Res Methods 46(2):596–610

    Article  Google Scholar 

  30. Mehrabian A, Russell JA (1974) An approach to environmental psychology the. MIT Press, Cambridge

    Google Scholar 

  31. Nardelli M, Valenza G, Greco A, Lanata A, Scilingo EP (2015) Recognizing emotions induced by affective sounds through heart rate variability. IEEE Trans Affect Comput 6(4):385–394

    Article  Google Scholar 

  32. Nasoz F, Alvarez K, Lisetti CL, Finkelstein N (2004) Emotion recognition from physiological signals using wireless sensors for presence technologies. Cog Technol Work 6(1):4–14

    Article  Google Scholar 

  33. Patrão B, Pedro S, Menezes P (2016) Human emotions and physiological signals: a classroom experiment. Int J Online Eng 12(4):37–39

    Article  Google Scholar 

  34. Picard RW (1999) Affective computing for HCI. In: HCI (1), Citeseer, pp 829–833.

  35. Picard RW, Vyzas E, Healey J (2001) Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Trans Patt Anal Mach Intell 23(10):1175–1191

    Article  Google Scholar 

  36. Plutchik R (1980) A general psychoevolutionary theory of emotion. In: Theories of Emotion. Elsevier, New York, pp 3–33

  37. Quiroz JC, Geangu E, Yong MH (2018) Emotion Recognition Using Smart Watch Sensor data: mixed-Design Study. JMIR Mental Health 5(3):e10153

    Article  Google Scholar 

  38. Raglio A (2015) Music therapy interventions in Parkinson’s disease: the state-of-the-art. Front Neurol 6:185

    Article  Google Scholar 

  39. Russell JA (1980) A circumplex model of affect. J Person Social Psychol 39 (6):1161

    Article  Google Scholar 

  40. Saibene A, Gasparini F El Yacoubi S., Bagnoli F, Pacini G (eds) (2019) Cognitive and Physiological Response for Health Monitoring in an Ageing population: A Multi-modal System, vol 11938. Springer International Publishing, Cham

  41. Sano A, Picard RW (2013) Stress recognition using wearable sensors and mobile phones. In: 2013 Humaine association conference on affective computing and intelligent interaction. IEEE, Los Alamitos, pp 671–676

  42. Sarsenbayeva Z, van Berkel N, Hettiachchi D, Jiang W, Dingler T, Velloso E, Kostakos V, Goncalves J (2019) Measuring the effects of stress on mobile interaction. Proceedings of the ACM on Interactive, Mobile. Wearable Ubiquitous Technol 3(1):1–18

    Article  Google Scholar 

  43. Serrano JPD, Soltez JMA, Pascual RKC, Castillo JCD, Torres JL, Cruz FRG (2018) Portable Stress Level Detector based on Galvanic Skin Response, Heart Rate, and Body Temperature. In: 2018 IEEE 10Th international conference on humanoid, nanotechnology, information technology, communication and control, environment and management (HNICEM). IEEE, Los Alamitos, pp 1–5

  44. Setz C, Arnrich B, Schumm J, La Marca R, Tröster G, Ehlert U (2009) Discriminating stress from cognitive load using a wearable EDA device. IEEE Trans Inform Technol Biomed 14(2):410–417

    Article  Google Scholar 

  45. Sriramprakash S, Prasanna VD, Murthy OR (2017) Stress detection in working people. Procedia Comput Sci 115:359–366

    Article  Google Scholar 

  46. Spherison Sound6D. https://www.spherison.it/. Accessed: 2020-01-10

  47. Subhani AR, Kamel N, Saad MNM, Nandagopal N, Kang K, Malik AS (2018) Mitigation of stress: new treatment alternatives. Cognitive Neurodynamics 12(1):1–20

    Article  Google Scholar 

  48. Subramanian R, Wache J, Abadi MK, Vieriu RL, Winkler S, Sebe N (2016) ASCERTAIN: Emotion And personality recognition using commercial sensors. IEEE Trans Affect Comput 9(2):147–160

    Article  Google Scholar 

  49. Trochidis K, Lui S (2015) Modeling affective responses to music using audio signal analysis and physiology. In: International symposium on computer music multidisciplinary research. Springer, New York, pp 346–357

  50. van Bruggen-Rufi M, Roos R (2015) The effect of music therapy for patients with Huntington’s disease: A systematic literature review. J Liter Art Stud 5(1):30–40

    Google Scholar 

  51. Vargha A, Delaney HD (2000) A critique and improvement of the CL common language effect size statistics of mcGraw and Wong. J Educ Behav Stat 25(2):101–132

    Google Scholar 

  52. Wagner J, Kim J, André E (2005) From physiological signals to emotions: Implementing and comparing selected methods for feature extraction and classification. In: 2005 IEEE International conference on multimedia and expo. IEEE, Los Alamitos, pp 940–943

  53. Watson D, Tellegen A (1985) Toward a consensual structure of mood. Psychol Bull 98(2):219

    Article  Google Scholar 

  54. Wohlin C, Runeson P, Höst M., Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in software engineering Springer Science & Business Media

  55. Zangróniz R, Martínez-rodrigo A, Pastor JM, Ló pez MT, Fernández-Caballero A (2017) Electrodermal activity sensor for classification of calm/distress condition. Sensors 17(10):2324

  56. Zubair M, Yoon C (2020) Multilevel mental stress detection using ultra-short pulse rate variability series. Biomed Signal Process Control 57:101736

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by the FONDAZIONE CARIPLO “LONGEVICITY-Social Inclusion for a Elderly through Walkability” (Rif. 2017-0938). We want to give our thanks to Eleonora Pizzi, a master student at the University of Milano-Bicocca, for her supporting work during the experimentation.

Author information

Authors and Affiliations

  1. Università di Milano-Bicocca, Dipartimento di Informatica, Milan, Italy

    Francesca Gasparini, Marta Giltri & Stefania Bandini

  2. RCAST - Research Center for Advanced Science & Technology, The University of Tokyo, Tokyo, Japan

    Stefania Bandini

Authors
  1. Francesca Gasparini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marta Giltri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefania Bandini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francesca Gasparini.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gasparini, F., Giltri, M. & Bandini, S. Discriminating affective state intensity using physiological responses. Multimed Tools Appl 79, 35845–35865 (2020). https://doi.org/10.1007/s11042-020-09114-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09114-y

Keywords

Search

Navigation