Skip to main content
SpringerLink
Log in
Book cover

Workshop on Computational Models of Natural Argument

International Workshop on Empathic Computing

International Workshop on Empathic Computing

CMNA 2015, IWEC 2015, IWEC 2014: Principles and Practice of Multi-Agent Systems pp 74–85Cite as

Neural Prediction of the User’s Mood from Visual Input

  • Conference paper
  • First Online:

  • 514 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9935))

Abstract

Affect-adaptive systems mutate their behavior according to the user’s affective state. In many cases, such affective state is to be detected in a non-obtrusive way, i.e. through sensing that does not require the user to provide the system explicit input, e.g., video sensors. However, user affect recognition from video is frequently tuned to detect instantaneous emotional states, rather than longer term and more constant affective states such as mood. In this paper, we propose a non-linear computational model for bridging the gap between the recognized emotions of a person captured by a video and the overall mood of the person. For the experimental validation, emotions and mood are human annotations on an affective visual database that we created on purpose. Based on features describing peculiarities and changes in the user’s emotional state, our system is able to predict the corresponding mood well above chance and more accurately than existing models.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Bradley, M.M., Peter, J.L.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)

    Article  Google Scholar 

  • Broekens, J., Brinkman, W.P.: AffectButton: towards a standard for dynamic affective user feedback. In: International Conference on Affective Computing & Intelligent Interaction (2009)

    Google Scholar 

  • Cohen, I., et al.: Facial expression recognition from video sequences: temporal and static modeling. Comput. Vis. Image Underst. 91(1), 160–187 (2003)

    Article  Google Scholar 

  • Cowie, R., McKeown, G.: Statistical analysis of data from initial labelled database and recommendations for an economical coding scheme (2006)

    Google Scholar 

  • Cowie, R., Sawey, M.: GTrace-General trace program from Queen’s Belfast (2011)

    Google Scholar 

  • Douglas-Cowie, E., et al.: The HUMAINE database: addressing the collection and annotation of naturalistic and induced emotional data. In: Paiva, A., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 488–500. Springer, Heidelberg (2007). doi:10.1007/978-3-540-74889-2_43

    Chapter  Google Scholar 

  • Ekman, P.: An argument for basic emotions. Cogn. Emot. 6(3-4), 169–200 (1992)

    Article  Google Scholar 

  • Ekman, P.: Basic emotions. Handb. Cogn. Emot. 98, 45–60 (1999)

    Google Scholar 

  • Gebhard, P.: ALMA – a layered model of affect. In: International Conference on Artificial Intelligent and Multi-Agent Systems (AAMAS) (2005)

    Google Scholar 

  • Gunes, H., Piccardi, M.: Bi-modal emotion recognition from expressive face and body gestures. J. Netw. Comput. Appl. 30, 1334–1345 (2007)

    Article  Google Scholar 

  • Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1), 489–501 (2006)

    Article  Google Scholar 

  • Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: IEEE International Joint Conference on Neural Networks (2004)

    Google Scholar 

  • Jenkins, J.M., Oatley, K., Stein, N.L.: Human Emotions: A Reader. Blackwell, Oxford (1998)

    Google Scholar 

  • Katsimerou, C., Redi, J.A., Heynderickx, I.: A computational model for mood recognition. In: Dimitrova, V., et al. (eds.) UMAP 2014. LNCS, vol. 8538, pp. 122–133. Springer, Heidelberg (2014). doi:10.1007/978-3-319-08786-3_11

    Google Scholar 

  • Kleinsmith, A., Bianchi-Berthouze, N.: Affective body expression perception and recognition: a survey. Trans. Affect. Comput. 4, 15–33 (2013)

    Article  Google Scholar 

  • Lane, A.M., Terry, P.C.: The nature of mood: development of a conceptual model with a focus on depression. J. Appl. Sport Psychol. 12(1), 16–33 (2000)

    Article  Google Scholar 

  • McKeown, G., Valstar, M.F., Cowie, R., Pantic, M.: The SEMAINE corpus of emotionally coloured character interactions. In: IEEE International Conference on Multimedia and Expo (ICME) (2010)

    Google Scholar 

  • Mehrabian, A.: Pleasure-arousal-dominance: a general framework for describing and measuring individual differences in temperament. Curr. Psychol. 14, 261–292 (1996)

    Article  MathSciNet  Google Scholar 

  • Metallinou, A., Katsamanis, A., Narayanan, S.: Tracking continuous emotional trends of participants during affective dyadic interactions using body language and speech information. Image Vis. Comput. 31(2), 137–152 (2013)

    Article  Google Scholar 

  • Metallinou, A., Narayanan, S.: Annotation and processing of continuous emotional attributes: challenges and opportunities. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) (2013)

    Google Scholar 

  • Metallinou, A., et al.: Context-sensitive learning for enhanced audiovisual emotion classification. IEEE Trans. Affect. Comput. 3(2), 184–198 (2012)

    Article  Google Scholar 

  • Morris, W.N.: Some thoughts about mood and its regulation. Psychol. Inq. 11, 200–202 (2000)

    Google Scholar 

  • Nicolaou, M., Gunes, H., Pantic, M.: Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space. IEEE Trans. Affect. Comput. 2, 92–105 (2011)

    Article  Google Scholar 

  • Nicolaou, M.A., Pavlovic, V., Pantic, M.: Dynamic probabilistic CCA for analysis of affective behaviour. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7578, pp. 98–111. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33786-4_8

    Chapter  Google Scholar 

  • Oatley, K., Johnson-Laird, P.N.: Towards a cognitive theory of emotions. Cogn. Emot. 1(1), 29–50 (1987)

    Article  Google Scholar 

  • Rusell, J.: A circumplex model of affect. Pers. Soc. Psychol. 39, 1161–1178 (1980)

    Article  Google Scholar 

  • Russell, J.A.: Core affect and the psychological construction of emotion. Psychol. Rev. 110(1), 145 (2003)

    Article  Google Scholar 

  • Sigal, L., Fleet, D.J., Troje, N.F., Livne, M.: Human attributes from 3D pose tracking. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6313, pp. 243–257. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15558-1_18

    Chapter  Google Scholar 

  • Thayer, R.E.: The Origin of Everyday Moods: Managing Energy, Tension, and Stress. Oxford University Press, Oxford (1996)

    Google Scholar 

  • Thrasher, M., Zwaag, M.D., Bianchi-Berthouze, N., Westerink, J.H.D.M.: Mood recognition based on upper body posture and movement features. In: D’Mello, S., Graesser, A., Schuller, B., Martin, J.-C. (eds.) ACII 2011. LNCS, vol. 6974, pp. 377–386. Springer, Heidelberg (2011). doi:10.1007/978-3-642-24600-5_41

    Chapter  Google Scholar 

  • Valstar, M., et al.: AVEC 2013: the continuous audio/visual emotion and depression recognition challenge. In: 3rd ACM International Workshop on Audio/Visual Emotion Challenge (2013)

    Google Scholar 

  • Västfjäll, D.: Emotion induction through music: a review of the musical mood induction procedure. Music Sci. 5(1), 173–211 (2002)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Multimedia Computing Group, Delft University of Technology, Delft, The Netherlands

    Christina Katsimerou & Judith A. Redi

Authors
  1. Christina Katsimerou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Judith A. Redi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christina Katsimerou .

Editor information

Editors and Affiliations

  1. Università degli Studi di Torino, Turin, Italy

    Matteo Baldoni

  2. Università degli Studi di Torino, Turin, Italy

    Cristina Baroglio

  3. Utrecht University, Utrecht, The Netherlands

    Floris Bex

  4. University of Liverpool, Liverpool, United Kingdom

    Floriana Grasso

  5. University of North Carolina Greensboro, Greensboro, North Carolina, USA

    Nancy Green

  6. University of Wollongong, Wollongong, New South Wales, Australia

    Mohammad-Reza Namazi-Rad

  7. Osaka University, Osaka, Japan

    Masayuki Numao

  8. De La Salle University, Manila, Philippines

    Merlin Teodosia Suarez

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Katsimerou, C., Redi, J.A. (2016). Neural Prediction of the User’s Mood from Visual Input. In: Baldoni, M., et al. Principles and Practice of Multi-Agent Systems. CMNA IWEC IWEC 2015 2015 2014. Lecture Notes in Computer Science(), vol 9935. Springer, Cham. https://doi.org/10.1007/978-3-319-46218-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-46218-9_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-46217-2

  • Online ISBN: 978-3-319-46218-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation