Skip to main content
SpringerLink
Log in

Evaluating a Computational Model of Emotion

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

Spurred by a range of potential applications, there has been a growing body of research in computational models of human emotion. To advance the development of these models, it is critical that we evaluate them against the phenomena they purport to model. In this paper, we present one method to evaluate an emotion model that compares the behavior of the model against human behavior using a standard clinical instrument for assessing human emotion and coping. We use this method to evaluate the Emotion and Adaptation (EMA) model of emotion Gratch and Marsella. The evaluation highlights strengths of the approach and identifies where the model needs further development.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. J. Gratch and S. Marsella, “Tears and fears: Modeling emotions and emotional behaviors in synthetic agents”, presented at Fifth International Conference on Autonomous Agents, Montreal, Canada, 2001.

  2. S. Marsella and J. Gratch, “The Modeling coping behaviors in virtual humans: Don’t worry, be happy”, presented at Second International Joint Conference on Autonomous Agents and Multi-agent Systems, Melbourne, Australia, 2003.

  3. J. Gratch S. Marsella (2004) ArticleTitle“A domain independent framework for modeling emotion” J. Cognitive Sys. Res 5 269–306 Occurrence Handle10.1016/j.cogsys.2004.02.002

    Article  Google Scholar 

  4. C. Conati (2002) ArticleTitle“Probabilistic assessment of user’s emotions in educational games” J. Appl. Artif. Intelligence, special issue on “Merging Cognition and Affect in HCI” 16 555–575

    Google Scholar 

  5. C. Conati and H. MacLaren, “Evaluating a probabilistic model of student affect,” presented at Seventh International Conference on Intelligent Tutoring Systems, Maceio, Brazil, 2004.

  6. S. Marsella, W. L. Johnson, and C. LaBore, “Interactive pedagogical drama,” presented at Fourth International Conference on Autonomous Agents, Montreal, Canada, 2000.

  7. B. O. Rothbaum L. F. Hodges R. Alarcon D. Ready F. Shahar K. Graap J. Pair P. Hebert B. Wills D. Baltzell (1999) ArticleTitle“Virtual environment exposure therapy for PTSD vietnam veterans: A case study” J. Trauma Stress 12 263–272 Occurrence Handle10.1023/A:1024772308758 Occurrence Handle1:STN:280:DyaK1MzgsF2hug%3D%3D Occurrence Handle10378165

    Article  CAS  PubMed  Google Scholar 

  8. J. C. Lester B. A. Stone G. D. Stelling (1999) ArticleTitle“Lifelike pedagogical agents for mixed-initiative problem solving in constructivist learning environments” User Model User-Adapted Instruction. 9 1–44 Occurrence Handle10.1023/A:1008374607830

    Article  Google Scholar 

  9. K. Ryokai C. Vaucelle J. Cassell (2003) ArticleTitle“Virtual Peers as Partners in Storytelling and Literacy Learning” Journal of Computer Assisted Learning 19 IssueID2 195–208 Occurrence Handle10.1046/j.0266-4909.2003.00020.x

    Article  Google Scholar 

  10. E. Shaw W. L. Johnson R. Ganeshan (1999) “Pedagogical agents on the web”, presented at Proceedings of the Third International Conference on Autonomous Agents Seattle WA

    Google Scholar 

  11. E. André T. Rist S. V. Mulken M. Klesen (2000) “The automated design of believable dialogues for animated presentation teams” J. Cassell J. Sullivan S. Prevost E. Churchill (Eds) Embodied Conversational Agents MIT Press Cambridge, MA 220–255

    Google Scholar 

  12. J. Cassell T. Bickmore L. Campbell H. Vilhjálmsson H. Yan (2000) “Human conversation as a system framework: Designing embodied conversational agents” J. Cassell J. Sullivan S. Prevost E. Churchill (Eds) Embodied Conversational Agents MIT Press Boston 29–63

    Google Scholar 

  13. B. G. Silverman, M. Johns, K. O’Brien, R. Weaver, and J. Cornwell, “Constructing Virtual Asymmetric Opponents from Data and Models in the Literature: Case of Crowd Rioting,” 11th Conference on Computer-Generated Forces and Behavior Representation, Orlanda FL, pp. 97–105, 2002.

  14. J. Gratch S. Marsella (2003) ArticleTitle“Fight the way you train: The role and limits of emotions in training for combat” Brown J. World Affairs 10 IssueID1 63–76

    Google Scholar 

  15. S. Marsella, D. Pynadath, and S. Read, “PsychSim: Agent-based modeling of social interactions and influence,” presented at International Conference on Cognitive Modeling, 2004.

  16. M. Minsky (1986) The Society of Mind Simon and Schuster New York

    Google Scholar 

  17. H. A. Simon (1967) ArticleTitle“Motivational and emotional controls of cognition” Psychological Rev 74 29–39 Occurrence Handle1:STN:280:CCiB38%2FpvFQ%3D

    CAS  Google Scholar 

  18. A. R. Damasio (1994) Descartes’ Error: Emotion, Reason, and the Human Brain Avon Books New York

    Google Scholar 

  19. K. Oatley P. N. Johnson-Laird (1987) ArticleTitle“Towards a Cognitive Theory of Emotions” Cognition and Emotion 1 IssueID1 29–50

    Google Scholar 

  20. A. Sloman M. Croucher (1981) “Why robots will have emotions”, presented at International Joint Conference on Artificial Intelligence Canada Vancouver

    Google Scholar 

  21. C. Lisetti P. Gmytrasiewicz (2002) ArticleTitle“Can a rational agent afford to be affectless? A formal approach” Appl. Artif. Intelligence 16 577–609 Occurrence Handle10.1080/08839510290030408

    Article  Google Scholar 

  22. R. Frank (1988) Passions with Reason: The Strategic Role of the Emotions W. W. Norton New York, NY

    Google Scholar 

  23. A. R. Mele (2001) Self-Deception Unmasked Princeton University Press Princeton, NJ

    Google Scholar 

  24. W. Mao and J. Gratch, “Social judgment in multiagent interactions”, presented at Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004.

  25. J. Gratch, J. Rickel, E. André, J. Cassell, E. Petajan, and N. Badler, “Creating interactive virtual humans: Some assembly required”, in IEEE Intelligent Syst., vol. July/August, pp. 54–61, 2002.

  26. J. Rickel, S. Marsella, J. Gratch, R. Hill, D. Traum, and W. Swartout, “Toward a new generation of virtual humans for interactive experiences”, in IEEE Intelligent Syst., vol. July/August, pp. 32–38 2002.

  27. S. Marsella, W. L. Johnson, and C. LaBore, “Interactive pedagogical drama for health interventions”, presented at Conference on Artificial Intelligence in Education, Sydney, Australia, 2003.

  28. M. Cavazza, F. Charles, and S. J. Mead, “Agents’ interaction in virtual storytelling”, presented at Third International Workshop on Intelligent Virtual Agents, 2001.

  29. C. L. Lisetti D. Schiano (2000) ArticleTitle“Facial expression recognition: Where human-computer interaction, artificial intelligence, and cognitive science intersect” Pragma Cognition 8 185–235

    Google Scholar 

  30. R. W. Picard (1997) Affective Computing MIT Press Cambridge, MA

    Google Scholar 

  31. S. van Mulken, E. André, and J. Muller, “The persona effect: how substantial is it,” presented at Human Computer Interaction Conference, Berlin, 1998.

  32. W. Swartout, R. Hill, J. Gratch, W. L. Johnson, C. Kyriakakis, C. LaBore, R. Lindheim, S. Marsella, D. Miraglia, B. Moore, J. Morie, J. Rickel, M. Thiebaux, L. Tuch, R. Whitney, and J. Douglas, “Toward the holodeck: Integrating graphics, sound, character and story,” presented at Fifth International Conference on Autonomous Agents, Montreal, Canada, 2001.

  33. R. Lazarus (1991) Emotion and Adaptation Oxford University Press NY

    Google Scholar 

  34. J. Velásquez, “When robots weep: Emotional memories and decision-making.,” presented at 15th National Conference on Artificial Intelligence, Madison, WI, 1998.

  35. C. A. Smith R. Lazarus (1990) “Emotion and Adaptation” L. A. Pervin (Eds) Handbook of Personality: Theory and research Guilford Press NY 609–637

    Google Scholar 

  36. E. Peacock P. Wong (1990) ArticleTitle“The stress appraisal measure (SAM): A multidimensional approach to cognitive appraisal” Stress Medic 6 227–236

    Google Scholar 

  37. J. Blythe (1999) ArticleTitle“Decision Theoretic Planning” in AI Magazine 20 IssueID2 37–54

    Google Scholar 

  38. B. Grosz S. Kraus (1996) ArticleTitle“Collaborative plans for complex group action” Artif. Intelligence 86 IssueID2 269–357 Occurrence Handle10.1016/0004-3702(95)00103-4

    Article  Google Scholar 

  39. K. G. Shaver (1985) The Attribution of Blame: Causality, Responsibility, and Blameworthiness Springer-Verlag NY

    Google Scholar 

  40. A. Ortony G. Clore A. Collins (1988) The Cognitive Structure of Emotions Cambridge University Press Cambridge

    Google Scholar 

  41. J. Gratch S. Marsella (2005) ArticleTitle“Lessons from Emotion Psychology for the Design of Lifelike Characters” Applied Artif Intelligence 19 IssueID3–4 215–233

    Google Scholar 

  42. C. Pelachaud, V. Carofiglio, B. D. Carolis, F. d. Rosis, and I. Poggi, “First international joint conference on autonomous agents and multiagent systems”, presented at Embodied Contextual Agent in Information Delivering Application, Bologna, Italy, 2002.

  43. M. Klesen (2005) ArticleTitle“Using theatrical concepts for role-plays with educational agents” Appl. Artif. Intelligence Special Issue “Educational Agents - Beyond Virtual Tutors” 19 IssueID3–4 413–431

    Google Scholar 

  44. H. Nakanishi S. Shimizu K. Isbister (2005) ArticleTitle“Sensitizing social agents for virtual training” Appl. Artif. Intelligence Special Issue “Educational Agents - Beyond Virtual Tutors” 19 IssueID3–4 393–412

    Google Scholar 

  45. A. Cowell and K. M. Stanney, “Embodiement and interaction guidelines for designing credible, trustworthy embodied conversational agents”, presented at Intelligent Virtual Agents, Kloster Irsee, Germany, 2003.

  46. J. C. Lester S. G. Towns C. B. Callaway J. L. Voerman P. J. FitzGerald (2000) “Deictic and emotive communication in animated pedagogical agents” J. Cassell S. Prevost J. Sullivan E. Churchill (Eds) Embodied Conversational Agents MIT Press Cambridge 123–154

    Google Scholar 

  47. G. Biswas, D. Schwartz, K. Leelawong, N. Vye, and TAG-V, “Learning by teaching. A new agent paradigm for educational software,” Appl. Artif. Intelligence special Issue “Educ. Agents - Beyond Virtual Tutors”, vol. 19, 2005.

  48. C. Elliott, “The affective reasoner: A process model of emotions in a multi-agent system,” Ph.D Dissertation, Northwestern University Institute for the Learning Sciences, Northwestern, IL, vol. 32, 1992.

  49. M. S. ElNasr J. Yen T. Ioerger (2000) ArticleTitle“FLAME: Fuzzy Logic Adaptive Model of Emotions” Autonomous Agents and Multi-Agent Systems 3 219–257 Occurrence Handle10.1023/A:1010030809960

    Article  Google Scholar 

  50. D. Moffat and N. Frijda, “Where there’s a Will there’s an agent,” presented at Workshop on Agent Theories, Architectures and Languages, 1995.

  51. J. L. Armony D. Servan-Schreiber J. D. Cohen J. E. LeDoux (1997) ArticleTitle“Computational modeling of emotion: Explorations through the anatomy and physiology of fear conditioning” Trends in Cognitive Sci 1 28–34 Occurrence Handle10.1016/S1364-6613(97)01007-3

    Article  Google Scholar 

  52. D. Cañamero, “Modeling motivations and emotions as a basis for intelligent behavior”, presented at International Conference on Autonomous Agents, Marina del Rey, CA, 1997.

  53. R. C. Arkin (2005) “Moving up the food chain: Motivation and Emotion in behavior-based robots” J. Fellous M. Arbib (Eds) Who Needs Emotions: The Brain Meets the Robot Oxford University Press New York

    Google Scholar 

  54. J. R. Busemeyer J. T. Townsend J. C. Stout (2002) “Motivational underpinnings of utility in decision making: Decision field theory analysis of deprivation and satiation” S. Moore (Eds) Emotional Cognition John Benjamins Amsterdam

    Google Scholar 

  55. P. Gmytrasiewicz and C. Lisetti, “Using decision theory to formalize emotions for multi-agent systems,” presented at Second ICMAS-2000 Workshop on Game Theoretic and Decision Theoretic Agents, Boston, 2000.

  56. H. Prendinger, S. Mayer, J. Mori, and M. Ishizuka, “Persona effect revisited,” presented at Intelligent Virtual Agents, Kloster Irsee, Germany, 2003.

  57. N. C. Kramer, B. Tietz, and G. Bente, “Effects of embodied interface agents and their gestural activity,” presented at Intelligent Virtual Agents, Kloster Irsee, Germany, 2003.

  58. M. Scheutz and A. Sloman, “Affect and agent control: experiments with simple affective states,” presented at IAT, 2001.

  59. M. Perrez M. Reicherts (1992) Stress, Coping, and Health Hogrefe and Huber Publishers Seattle, WA

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Creative Technology, University of Southern California, 13274 Fiji Way, Marina del Rey, CA, 90292, USA

    Jonathan Gratch

  2. Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Marina del Rey, CA, 90292, USA

    Stacy Marsella

Authors
  1. Jonathan Gratch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stacy Marsella
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jonathan Gratch.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gratch, J., Marsella, S. Evaluating a Computational Model of Emotion. Auton Agent Multi-Agent Syst 11, 23–43 (2005). https://doi.org/10.1007/s10458-005-1081-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-005-1081-1

Keywords

Search

Navigation