Skip to main content
SpringerLink
Log in

Automatic Recognition of Experienced Emotional State from Body Movement

  • Conference paper
  • First Online:
Human-Computer Interaction. Theory, Methods and Tools (HCII 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12762))

Included in the following conference series:

  • 3522 Accesses

Abstract

Although body movement carries information about a person’s emotional state, this modality is not widely used in automatic emotion measurement systems. With this paper, we address the question of the automatic recognition of a person’s affective state by analyzing the way a person moves. We present the approach which was used to build a classifier of experienced, non-acted emotions. To collect the data to train and validate the classifier, a controlled laboratory study was conducted. During the study, the music mood induction procedure was used to evoke different emotions in the participants. The participant’s movement was recorded using a depth sensor, two accelerometers, and two electromyography sensors. An accuracy of 43% was achieved to recognize four emotion classes, corresponding to four quadrants of the valence-arousal space. For the participants with lower dance or movement proficiency, recognition was more accurate. The same was discovered for the participants with higher comfort levels of moving in front of a camera. The findings show the potential for the automatic analysis of a person’s body movements to gather information about an affective state.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alaoui, S.F., Caramiaux, B., Serrano, M., Bevilacqua, F.: Movement qualities as interaction modality. In: Proceedings of the Designing Interactive Systems Conference, pp. 761–769. ACM (2012)

    Google Scholar 

  2. Appelhans, B.M., Luecken, L.J.: Heart rate variability as an index of regulated emotional responding. Rev. Gen. Psychol. 10(3), 229 (2006)

    Article  Google Scholar 

  3. Balasubramanian, S., Melendez-Calderon, A., Burdet, E.: A robust and sensitive metric for quantifying movement smoothness. IEEE Trans. Biomed. Eng. 59(8), 2126–2136 (2012)

    Article  Google Scholar 

  4. Bernhardt, D., Robinson, P.: Detecting emotions from everyday body movements. Presenccia PhD Sym., Barcelona (2007)

    Google Scholar 

  5. Blom, L.A., Chaplin, L.T.: The Intimate act of Choreography. University of Pittsburgh Pre (1982)

    Google Scholar 

  6. Boone, R.T., Cunningham, J.G.: Children’s decoding of emotion in expressive body movement: the development of cue attunement. Dev. Psychol. 34(5), 1007 (1998)

    Article  Google Scholar 

  7. Bouchard, D., Badler, N.: Semantic segmentation of motion capture using Laban movement analysis. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 37–44. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74997-4_4

    Chapter  Google Scholar 

  8. Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)

    Article  Google Scholar 

  9. Camurri, A., Lagerlöf, I., Volpe, G.: Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques. Int. J. Hum. Comput. Stud. 59(1), 213–225 (2003)

    Article  Google Scholar 

  10. Camurri, A., et al.: The dancer in the eye: towards a multi-layered computational framework of qualities in movement. In: Proceedings of the 3rd International Symposium on Movement and Computing, p. 6. ACM (2016)

    Google Scholar 

  11. Castellano, G., Villalba, S.D., Camurri, A.: Recognising human emotions from body movement and gesture dynamics. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 71–82. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74889-2_7

    Chapter  Google Scholar 

  12. Chandrashekar, G., Sahin, F.: A survey on feature selection methods. Comput. Electr. Eng. 40(1), 16–28 (2014)

    Article  Google Scholar 

  13. Ekman, P., Cordaro, D.: What is meant by calling emotions basic. Emot. Rev. 3(4), 364–370 (2011)

    Article  Google Scholar 

  14. Ekman, P., Levenson, R.W., Friesen, W.V.: Autonomic nervous system activity distinguishes among emotions. American Association for the Advancement of Science (1983)

    Google Scholar 

  15. Fdili Alaoui, S., Françoise, J., Schiphorst, T., Studd, K., Bevilacqua, F.: Seeing, sensing and recognizing Laban movement qualities. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 4009–4020. ACM (2017)

    Google Scholar 

  16. Juslin, P.N., Laukka, P.: Expression, perception, and induction of musical emotions: a review and a questionnaire study of everyday listening. J. New Music Res. 33(3), 217–238 (2004)

    Article  Google Scholar 

  17. Kleinginna, P.R., Kleinginna, A.M.: A categorized list of emotion definitions, with suggestions for a consensual definition. Motiv. Emot. 5(4), 345–379 (1981)

    Article  Google Scholar 

  18. Lane, R.D., McRae, K., Reiman, E.M., Chen, K., Ahern, G.L., Thayer, J.F.: Neural correlates of heart rate variability during emotion. Neuroimage 44(1), 213–222 (2009)

    Article  Google Scholar 

  19. Li, M., Chai, Q., Kaixiang, T., Wahab, A., Abut, H.: EEG emotion recognition system. In: Takeda, K., Erdogan, H., Hansen, J.H.L., Abut, H. (eds.) In-Vehicle Corpus and Signal Processing for Driver Behavior, pp. 125–135. Springer, Boston (2009). https://doi.org/10.1007/978-0-387-79582-9_10

    Chapter  Google Scholar 

  20. Lien, J.J., Kanade, T., Cohn, J.F., Li, C.C.: Automated facial expression recognition based on FACS action units. In: Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, pp. 390–395. IEEE (1998)

    Google Scholar 

  21. Lisetti, C.: Affective computing (1998)

    Google Scholar 

  22. Lundqvist, L.O., Carlsson, F., Hilmersson, P., Juslin, P.N.: Emotional responses to music: experience, expression, and physiology. Psychol. Music 37(1), 61–90 (2009)

    Article  Google Scholar 

  23. Masuda, M., Kato, S.: Motion rendering system for emotion expression of human form robots based on Laban movement analysis. In: 2010 IEEE RO-MAN, pp. 324–329. IEEE (2010)

    Google Scholar 

  24. Mogenson, G.J., Jones, D.L., Yim, C.Y.: From motivation to action: functional interface between the limbic system and the motor system. Prog. Neurobiol. 14(2), 69–97 (1980)

    Article  Google Scholar 

  25. Nakasone, A., Prendinger, H., Ishizuka, M.: Emotion recognition from electromyography and skin conductance. In: Proceedings of the 5th International Workshop on Biosignal Interpretation, pp. 219–222 (2005)

    Google Scholar 

  26. Nwe, T.L., Foo, S.W., De Silva, L.C.: Speech emotion recognition using Hidden Markov Models. Speech Commun. 41(4), 603–623 (2003)

    Article  Google Scholar 

  27. Piana, S., Staglianò, A., Odone, F., Camurri, A.: Adaptive body gesture representation for automatic emotion recognition. ACM Trans. Interact. Intell. Syst. (TiiS) 6(1), 6 (2016)

    Google Scholar 

  28. Picard, R.W.: Affective computing for HCI. In: HCI (1), pp. 829–833 (1999)

    Google Scholar 

  29. Pollick, F.E., Paterson, H.M., Bruderlin, A., Sanford, A.J.: Perceiving affect from arm movement. Cognition 82(2), B51–B61 (2001)

    Article  Google Scholar 

  30. Robinson, M.D., Clore, G.L.: Belief and feeling: evidence for an accessibility model of emotional self-report. Psychol. Bull. 128(6), 934 (2002)

    Article  Google Scholar 

  31. Russell, J.A.: A circumplex model of affect. J. Pers. Soc. Psychol. 39(6), 1161 (1980)

    Article  Google Scholar 

  32. Serra, T.: Osceleton vol 1.2.1 [software], July 2011. https://github.com/Sensebloom/OSCeleton

  33. Sheets-Johnstone, M.: Emotion and movement. A beginning empirical-phenomenological analysis of their relationship. J. Conscious. Stud. 6(11–12), 259–277 (1999)

    Google Scholar 

  34. Sinclair, J., Taylor, P.J., Hobbs, S.J.: Digital filtering of three-dimensional lower extremity kinematics: an assessment. J. Hum. Kinet. 39(1), 25–36 (2013)

    Article  Google Scholar 

  35. Soleymani, M., Caro, M.N., Schmidt, E.M., Sha, C.Y., Yang, Y.H.: 1000 songs for emotional analysis of music. In: Proceedings of the 2nd ACM International Workshop on Crowdsourcing for Multimedia, pp. 1–6. ACM (2013)

    Google Scholar 

  36. Van den Stock, J., Righart, R., De Gelder, B.: Body expressions influence recognition of emotions in the face and voice. Emotion 7(3), 487 (2007)

    Article  Google Scholar 

  37. Västfjäll, D.: Emotion induction through music: a review of the musical mood induction procedure. Musicae Scientiae 5(1\(\_\)Suppl.), 173–211 (2001)

    Google Scholar 

Download references

Acknowledgement

We are grateful for Martin Burghart and Sonia Sobol’s help to finalize the manuscript.

Author information

Authors and Affiliations

  1. Quality and Usability Lab, Technische Universität Berlin, Berlin, Germany

    Jan-Niklas Voigt-Antons, Petr Devaikin & Tanja Kojić

  2. German Research Center for Artificial Intelligence (DFKI), Berlin, Germany

    Jan-Niklas Voigt-Antons

Authors
  1. Jan-Niklas Voigt-Antons
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Petr Devaikin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tanja Kojić
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jan-Niklas Voigt-Antons .

Editor information

Editors and Affiliations

  1. The Open University of Japan, Chiba, Japan

    Masaaki Kurosu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Voigt-Antons, JN., Devaikin, P., Kojić, T. (2021). Automatic Recognition of Experienced Emotional State from Body Movement. In: Kurosu, M. (eds) Human-Computer Interaction. Theory, Methods and Tools. HCII 2021. Lecture Notes in Computer Science(), vol 12762. Springer, Cham. https://doi.org/10.1007/978-3-030-78462-1_49

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-78462-1_49

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-78461-4

  • Online ISBN: 978-3-030-78462-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation