Skip to main content
SpringerLink
Log in

Towards Real-Time Affect Detection Based on Sample Entropy Analysis of Expressive Gesture

  • Conference paper
Affective Computing and Intelligent Interaction (ACII 2011)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 6974))

Abstract

Aiming at providing a solid foundation to the creation of future affect detection applications in HCI, we propose to analyze human expressive gesture by computing movement Sample Entropy (SampEn). This method provides two main advantages: (i) it is adapted to the non-linearity and non-stationarity of human movement; (ii) it allows a fine-grain analysis of the information encoded in the movement features dynamics. A realtime application is presented, implementing the SampEn method. Preliminary results obtained by computing SampEn on two expressive features, smoothness and symmetry, are provided in a video available on the web.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Borod, J., Haywood, C., Koff, E.: Neuropsychological aspects of facial asymmetry during emotional expression: A review of the normal adult literature. Neuropsychology Review 7(1), 41–60 (1997)

    Article  Google Scholar 

  2. Calvo, R., D’Mello, S.: Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing 1(1) (2010)

    Google Scholar 

  3. Camurri, A., Ferrentino, P.: Interactive environments for music and multimedia. Multimedia Systems 7(1), 32–47 (1999)

    Article  Google Scholar 

  4. Camurri, A., Volpe, G., De Poli, G., Leman, M.: Communicating Expressiveness and Affect in Multimodal Interactive Systems. IEEE Multimedia, 43–53 (2005)

    Google Scholar 

  5. Castellano, G., Mortillaro, M., Camurri, A., Volpe, G., Scherer, K.: Automated Analysis of Body Movement in Emotionally Expressive Piano Performances. Music Perception 26(2), 103–119 (2008)

    Article  Google Scholar 

  6. Douglas-Cowie, E., Campbell, N., Cowie, R., Roach, P.: Emotional speech: Towards a new generation of databases. Speech Communication 40(1-2), 33–60 (2003)

    Article  MATH  Google Scholar 

  7. Eckmann, J., Ruelle, D.: Ergodic theory of chaos and strange attractors. Reviews of Modern Physics 57(3), 617–656 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  8. Fitts, P.: The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology 47(6), 381–391 (1954)

    Article  Google Scholar 

  9. Gallaher, P.E.: Individual differences in nonverbal behavior: Dimensions of style. Journal of Personality and Social Psychology 63(1), 133–145 (1992)

    Article  Google Scholar 

  10. Glowinski, D., Bracco, F., Chiorri, C., Atkinson, A., Coletta, P., Camurri, A.: An investigation of the minimal visual cues required to recognize emotions from human upper-body movements. In: Proceedings of ACM International Conference on Multimodal Interfaces (ICMI), Workshop on Affective Interaction in Natural Environments (AFFINE). ACM, New York (2008)

    Google Scholar 

  11. Glowinski, D., Camurri, A., Volpe, G., Dael, N., Scherer, K.: Technique for automatic emotion recognition by body gesture analysis. In: Computer Vision and Pattern Recognition 2008. CVPR Workshops. IEEE Computer Society, Los Alamitos (2008)

    Google Scholar 

  12. Glowinski, D., Coletta, P., Volpe, G., Camurri, A., Chiorri, C., Schenone, A.: Multi-scale entropy analysis of dominance in social creative activities. In: Proceedings of the International Conference on Multimedia, pp. 1035–1038. ACM, New York (2010)

    Google Scholar 

  13. Glowinski, D., Dael, N., Camurri, A., Volpe, G., Mortillaro, M., Scherer, K.: Towards a minimal representation of affective gestures. IEEE Transactions on Affective Computing (99), 1–13 (2011)

    Google Scholar 

  14. Grassberger, P., Procaccia, I.: Measuring the strangeness of strange attractors. Physica D: Nonlinear Phenomena 9(1-2), 189–208 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  15. Hong, S., Newell, K.: Entropy conservation in the control of human action. Nonlinear Dynamics, Psychology, and Life Sciences 12(2), 163 (2008)

    Google Scholar 

  16. Johansson, G.: Visual perception of biological motion and a model for its analysis. Perception and Psychophysics 14, 201–211 (1973)

    Article  Google Scholar 

  17. Kim, J., Andre, E.: Four-Channel Biosignal Analysis and Feature Extraction for Automatic Emotion Recognition. In: Biomedical Engineering Systems and Technologies, pp. 265–277 (2009)

    Google Scholar 

  18. Kleinsmith, A., Bianchi-Berthouze, N., Steed, A.: Automatic recognition of non-acted affective postures. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics (99), 1–12 (2011)

    Google Scholar 

  19. Mehrabian, A.: Nonverbal Communication. Aldine (2007)

    Google Scholar 

  20. OpenNI, http://www.openni.org

  21. Packard, N., Crutchfield, J., Farmer, J., Shaw, R.: Geometry from a time series. Physical Review Letters 45(9), 712–716 (1980)

    Article  Google Scholar 

  22. Pantic, M., Pentland, A., Nijholt, A., Huang, T.: Human Computing and Machine Understanding of Human Behavior: A Survey. In: Huang, T.S., Nijholt, A., Pantic, M., Pentland, A. (eds.) ICMI/IJCAI Workshops 2007. LNCS (LNAI), vol. 4451, pp. 47–71. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  23. Pincus, S.: Approximate entropy as a measure of system complexity. Proceedings of the National Academy of Sciences of the United States of America 88(6), 2297 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  24. Ramdani, S., Seigle, B., Lagarde, J., Bouchara, F., Bernard, P.: On the use of sample entropy to analyze human postural sway data. Medical Engineering & Physics 31(8), 1023–1031 (2009)

    Article  Google Scholar 

  25. Richman, J., Moorman, J.: Physiological time-series analysis using approximate entropy and sample entropy. American Journal of Physiology- Heart and Circulatory Physiology 278(6), H2039 (2000)

    Google Scholar 

  26. Roether, C., Omlor, L., Giese, M.: Lateral asymmetry of bodily emotion expression. Current Biology 18(8), 329–330 (2008)

    Article  Google Scholar 

  27. Russell, J.A.: A circumplex model of affect. Journal of Personality and Social Psychology 39(6), 1161–1178 (1980)

    Article  Google Scholar 

  28. Savitzky, A., Golay, M.J.E.: Smoothing and differentiation of data by simplified least squares procedures. Analytical Chemistry 36(8), 1627–1639 (1964)

    Article  Google Scholar 

  29. Seely, A., Macklem, P.: Complex systems and the technology of variability analysis. Crit Care 8(6), R367–R384 (2004)

    Article  Google Scholar 

  30. Shannon, C., Weaver, W.: The mathematical theory of information (1949)

    Google Scholar 

  31. Takens, F.: Detecting strange attractors in turbulence. In: Dynamical Systems and Turbulence, Warwick 1980 pp. 366–381 (1980)

    Google Scholar 

  32. Todorov, E., Jordan, M.I.: Smoothness maximization along a predefined path accurately predicts the speed profiles of complex arm movements. Journal of Neurophysiology 80(2), 696–714 (1998)

    Google Scholar 

  33. Vinciarelli, A., Pantic, M., Bourlard, H.: Social signals, their function, and automatic analysis: a survey. In: Proceedings of the 10th International Conference on Multimodal Interfaces, pp. 61–68. ACM, New York (2008)

    Chapter  Google Scholar 

  34. Wallbott, H.G., Scherer, K.R.: Cues and channels in emotion recognition. Journal of Personality and Social Psychology 51(4), 690–699 (1986)

    Article  Google Scholar 

  35. Wallbott, H.: Bodily expression of emotion. Eur. J. Soc. Psychol. 28, 879–896 (1998)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. InfoMus Lab, DIST, University of Genova, Italy

    Donald Glowinski & Maurizio Mancini

Authors
  1. Donald Glowinski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maurizio Mancini
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Memphis, 202 Psychology Building, 38152, Memphis, TN, USA

    Sidney D’Mello  & Arthur Graesser  & 

  2. Technische Universität München, Arcisstraße 21, 80333, München, Germany

    Björn Schuller

  3. Laboratoire d’Informatique pour la Mécanique et les Sciences de l’Ingénieur (LIMSI-CNRS), Bâtiment 508, 91403, Orsay Cedex, France

    Jean-Claude Martin

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Glowinski, D., Mancini, M. (2011). Towards Real-Time Affect Detection Based on Sample Entropy Analysis of Expressive Gesture. In: D’Mello, S., Graesser, A., Schuller, B., Martin, JC. (eds) Affective Computing and Intelligent Interaction. ACII 2011. Lecture Notes in Computer Science, vol 6974. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24600-5_56

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-24600-5_56

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-24599-2

  • Online ISBN: 978-3-642-24600-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation