Skip to main content
SpringerLink
Log in

Empirically building and evaluating a probabilistic model of user affect

  • Original Paper
  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

Abstract

We present a probabilistic model of user affect designed to allow an intelligent agent to recognise multiple user emotions during the interaction with an educational computer game. Our model is based on a probabilistic framework that deals with the high level of uncertainty involved in recognizing a variety of user emotions by combining in a Dynamic Bayesian Network information on both the causes and effects of emotional reactions. The part of the framework that reasons from causes to emotions (diagnostic model) implements a theoretical model of affect, the OCC model, which accounts for how emotions are caused by one’s appraisal of the current context in terms of one’s goals and preferences. The advantage of using the OCC model is that it provides an affective agent with explicit information not only on which emotions a user feels but also why, thus increasing the agent’s capability to effectively respond to the users’ emotions. The challenge is that building the model requires having mechanisms to assess user goals and how the environment fits them, a form of plan recognition. In this paper, we illustrate how we built the predictive part of the affective model by combining general theories with empirical studies to adapt the theories to our target application domain. We then present results on the model’s accuracy, showing that the model achieves good accuracy on several of the target emotions. We also discuss the model’s limitations, to open the ground for the next stage of the work, i.e., complementing the model with diagnostic information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alessi, S.M., Trollip, S.R.: Multimedia for Learning: Methods and Development. 3rd edn. Allyn & Bacon, Needham Heights (2001)

  • Baker, R.S., Corbett, A.T., Koedinger, K.R., Wagner, A.Z.: Off-task behavior in the cognitive tutor classroom: when students “game the system”. In: CHI’04: Conference on Human Factors in Computing systems, pp. 383–390. Vienna, Austria (2004)

  • Bosma, W., André, E.: Exploiting emotions to disambiguate dialogue acts. In: IUI’04, International Conference on Intelligent User Interfaces, pp. 85–92. Madeira, Portugal (2004)

  • Chalfoun, P., Haffar, S., Frasson, C.: Predicting the emotional reaction of the learner with a machine learning technique. In: Workshop on Motivational and Affective Issues in ITS, ITS’06, International Conference on Intelligent Tutoring Systems. Jhongli, Taiwan (2006)

  • Cohen J.: Statistical Power Analysis for the Behavioral Sciences. 2nd edn. Lawrence Earlbaum Associates, Hillsdale (1988)

    Google Scholar 

  • Conati C.: Probabilistic assessment of users’ emotions in educational games. Special issue on “Merging Cognition and Affect in HCI”. J. Appl. Artifi. Intell. 16(7–8), 555–575 (2002)

    Article  Google Scholar 

  • Conati, C.: How to evaluate models of user affect? In: ADS’04, Tutorial and Research Workshop on Affective Dialog Systems, pp. 288–300. Kloster Irsee, Germany (2004)

  • Conati C., Klawe M.: Socially intelligent agents in educational games. In: Dautenhahn, K., Bond, A., Canamero, D., Edmonds, B. (eds) Socially Intelligent Agents—Creating Relationships with Computers and Robots, pp. 213–220. Kluwer Academic Publishers, Dordrecht (2002)

    Google Scholar 

  • Conati, C., Maclaren, H.: Evaluating a probabilistic model of student affect. In: ITS’04, International Conference on Intelligent Tutoring Systems, pp. 55–66. Maceio, Brazil (2004)

  • Conati, C., Zhao, X.: Building and evaluating an intelligent pedagogical agent to improve the effectivenesss of an education game. In: IUI’04, International Conference on Intelligent User Interfaces, pp. 6–13. Funchal, Madeira, Portugal (2004)

  • Conati, C., Chabbal, R., Maclaren, H.: A study on using biometric sensors for detecting user emotions in educational games. In: Workshop on “Assessing and Adapting to User Attitude and Affects: Why, When, and How?” in conjunction with UM’03, 9th International Conference on User Modeling, Pittsburgh, USA (2003)

  • Cordova D., Lepper M.: Intrinsic motivation and the process of learning: beneficial effects of contextualization, personalization, and choice. J. Educ. Psychol. 88, 715–730 (1996)

    Article  Google Scholar 

  • Costa P.T., McCrae R.R.: Four ways five factors are basic. Personal. Individ. Differ. 13, 653–665 (1992)

    Article  Google Scholar 

  • Craig S.D., Graesser A.C., Sullins J., Gholson B.: Affect and learning: an exploratory look into the role of affect in learning with auto tutor. J. Educ. Media 29, 241–250 (2004)

    Google Scholar 

  • Dean T., Kanazawa K.: A model for reasoning about persistance and causation. Comput. Intell. 5(3), 142–150 (1989)

    Article  Google Scholar 

  • D’Mello S.K., Craig S.D., Sullins J., Graesser A.C.: Predicting affective states expressed through an Emote–Aloud procedure from autotutor’s mixed-initiative dialogue. Int. J. Artifi. Intell. Educ. 16, 3–28 (2006)

    Google Scholar 

  • D’Mello, S.K., Chipman, P., Graesser, A.C.: Posture as a predictor of learner’s affective engagement. In: Proceedings of the 29th Annual Cognitive Science Society, pp. 571–576. Stresa, Italy (2006)

  • D’Mello S.K., Craig S.D., Witherspoon A.W., McDaniel B.T., Graesser A.C.: Automatic detection of learner’s affect from conversational cues. User Model. User-Adapt. Interact. 18(1–2), 45–80 (2008)

    Article  Google Scholar 

  • de Vincente, A., Pain, H.: Motivation self-report in ITS. In: AIED’99, 9th International Conference on Artificial Intelligence in Education, pp. 651–659. Le Mans, France (1999)

  • Dias, J., Paiva, A.: Feeling and reasoning: a computational model for emotional characters. In: Progress in Artificial Intelligence, 12th Portuguese Conference on Artificial Intelligence, EPIA 2005, pp. 127–140. Covilhã, Portugal (2005)

  • Elliot C., Rickel J., Lester J.C.: Lifelike pedagogical agents and affective computing: an exploratory synthesis. In: Wooldridge, M., Veloso, M. (eds) Artificial Intelligence Today, pp. 195–211. Springer, Berlin (1999)

    Chapter  Google Scholar 

  • Esuli, A., Fagni, T., Sebastiani, F., Aizerman, M.: MP-Boost: A multiple-pivot boosting algorithm and its application to text categorization. In: 13th International Symposium on String Processing and Information Retrieval (SPIRE’06), pp. 1–12. Glasgow, UK (2006)

  • Fisher R.A.: The Design of Experiment. Hafner, New York (1935)

    Google Scholar 

  • Gebhard, P.: ALMA—a layered model of affect. In: 4th International Joint Conference of Autonomous Agents & Multi-Agent Systems (AAMAS’05), pp. 29–36. Utrecht, The Netherlands (2005)

  • Gratch, J.: Emile: marshalling passions in training and education. In: 4th International Conference on Autonomous Agents, pp. 325–332. Barcelona, Spain (2000)

  • Gratch J., Marsella S.: A domain independent framework for modeling emotion. J. Cogn. Syst. Res. 5(4), 269–306 (2004)

    Article  Google Scholar 

  • Graziano W.G., Jensen-Campbell L.A., Finch J.F.: The self as a mediator between personality and adjustment. J. Personal. Soc. Psychol. 73, 392–404 (1997)

    Article  Google Scholar 

  • Healey J.A., Picard R.W.: Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans. Intell. Transp. Syst. 6(2), 156–166 (2005)

    Article  Google Scholar 

  • Heckerman D.: A tutorial on learning with Bayesian networks. In: Jordan, M. (eds) Learning in Graphical Models, pp. 301–354. MIT Press, Cambridge (1999)

    Google Scholar 

  • Hudlicka E., McNeese M.: Assessment of user affective and belief states for interface adaptation: application to an air force pilot. User Model. User Adapt. Interact. 12(1), 1–47 (2002)

    Article  MATH  Google Scholar 

  • Jameson A.: “Adaptive Interfaces and Agents”. In: Jacko, J.A., Sears, A. (eds) Human–Computer Interface Handbook, pp. 305–330. Erlbaum, Mahwah (2003)

    Google Scholar 

  • Kapoor, A., Picard, R.W.: Multimodal Affect Recognition in Learning Environments. In: 13th Annual ACM International Conference on Multimedia, pp. 677–682. Singapore (2005)

  • Kim, J., André, E.: Emotion recognition using physiological and speech signal in short-term observation. In: PIT’06, Perception and Interactive Techologies, pp. 53–64. Kloster Irsee, Germany (2006)

  • Klawe, M.: When does the use of computer games and other interactive multimedia software help students learn mathematics? In: Technology and NCTM Standards 2000 Conference, Arlington, VA (1998)

  • Lang P.J., Greenwald M.K., Bradley M.M., Hamm A.O.: Look at pictures: affective, facial, visceral, and behavioral reactions. Psychophysiology 30, 261–273 (1993)

    Article  Google Scholar 

  • Lee, J., Luchini, K., Michael, B., Norris, C., Solloway, E.: More than just fun and games: assessing the value of educational video games in the classroom. In: Proceedings of ACM SIGCHI 2004, pp. 1375–1378. Vienna, Austria (2004)

  • Lepper M., Woolverton M., Mumme D., Gurtner J.L.: Motivational techniques of expert human tutors: lessons for the design of computer-based tutors. In: Lajoie, S.P., Derry, S.J. (eds) Computers as Cognitive Tools, pp. 75–105. Lawrence Erlbaum Associates, New Jersey (1993)

    Google Scholar 

  • Li, X., Ji, Q.: Active affective state detection and user assistance. In: Workshop on “Assessing and Adapting to User Attitude and Affects: Why, When, and How?” in conjunction with UM’03, 9th International Conference on User Modeling. Pittsburgh, PA (2003)

  • Linnenbrink E.A., Pintrich P.R.: The Role of Motivational Beliefs in Conceptual Change. In: Limon, M., Mason, L. (eds) Reconsidering Conceptual Change: Issues in Theory and Practice, pp. 115–135. Kluwer Academic Publishers, Dordrecht (2002)

    Chapter  Google Scholar 

  • Litman, D.J., Forbes-Riley, K.: Predicting Student Emotions in Computer–Human Tutoring Dialogues. In: 42nd Annual Meeting of the Association for Computational Linguistics (ACL), pp. 352–359. Barcelona, Spain (2004)

  • Mandryk R.L., Inkpen K.M., Calvert T.W.: Using psychophysiological techniques to measure user experience with entertainment technologes. Special issue on User Experience. J. Behav. Inf. Technol. 25, 141–158 (2006)

    Article  Google Scholar 

  • Manske, M., Conati, C.: Modeling learning in educational games. In: AIED’05, 12th International Conference on AI in Education, pp. 411–418. Amsterdam (2005)

  • McQuiggan, S., Lee, S., Lester, J.: Early prediction of student frustration. In: Second Int. Conf. on Affective Computing and Intelligent Interactions, pp. 698–709. Lisbona, Portugal (2007)

  • Mehrabian A.: Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Curr. Psychol. 14, 261–292 (1996)

    Article  MathSciNet  Google Scholar 

  • Mitchell T.: Machine Learning. McGraw Hill, New York (1997)

    MATH  Google Scholar 

  • Ortony A., Clore G.L., Collins A.: The Cognitive Stucture of Emotions. Cambridge University Press, Cambridge (1988)

    Google Scholar 

  • Peter C., Herbon A.: Emotion representation and physiology assignments in digital systems. Interact. Comput. 18(2), 139–170 (2006)

    Article  Google Scholar 

  • Picard R.W., Vyzas E., Healey J.A.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)

    Article  Google Scholar 

  • Prendinger, H., Ishizuka, M.: SCREAM: scripting emotion-based agent minds, pp. 350–351. AAMAS, Bologna, Italy (2002)

  • Prendinger, H., Mori, J., Ishizuka, M.: Recognizing, modeling, and responding to users’ affective states. In: UM’05 10th International Conference on User Modeling, pp. 60–69. Edinburgh, UK (2005)

  • Rodrigo, M.T., Baker, R., D’Mello, S.T., Gonzales, et~al.: Comparing learners’ affect while using an intelligent tutoring system and a simulation problem solving game. In: Intelligent Tutoring Systems 2008, pp. 40–49. Montreal, Canada (2008)

  • Scheirer J., Fernandez R., Klein J., Picard R.W.: Frustrating the user on purpose: a step toward building an affective computer. Interact. Comput. 14(2), 93–118 (2002)

    Google Scholar 

  • Sebastiani F.: Machine learning in automated text categorization. ACM Comput. Surv. 34(1), 1–47 (2002)

    Article  Google Scholar 

  • Seif El-Nasr, M., Yen, J., Ioerger T.R.: FLAME—fuzzy logic adaptive model of emotions, pp. 219–257. Autonomous Agents and Multi-Agent Systems, Barcelona, Spain (2000)

  • Streit, M., Batliner, A., Portele, T.: Cognitive-model-based interpretation of emotions in a multi-model dialog system. In: ADS’04 Tutorial and Research Workshop on Affective Dialogue Systems, pp. 65–76. Kloster Irsee, Germany (2004)

  • Thayer Robert E.: The Biopsychology of Mood and Arousal. Oxford University Press, New York (1989)

    Google Scholar 

  • Van Eck R.: Building artificially intelligent learning games. In: Gibson, D., Aldrich, C., Prensky, M. (eds) Games and Simulations in Online Learning: Research and Development Frameworks, pp. 271–307. Information Science Pub, Hershey, PA (2007)

    Google Scholar 

  • Van Lehn K., Niu Z.: Bayesian student modeling, user interfaces and feedback: a sensitivity analysis. Int. J. Artifi. Intell. Educ. 12(2), 154–184 (2001)

    Google Scholar 

  • Vogel J.J., Greenwood-Ericksen A., Cannon-Bowers J., Bowers C.A.: Using virtual reality with and without gaming attributes for academic achievement. J. Res. Technol. Educ. 39(1), 105–118 (2006)

    Google Scholar 

  • Yannakakis G.N., Hallam J., Lund H.H.: Entertainment capture through heart rate activity in physical interactive playgrounds. User Model. User-Adapt. Interact. 18(1–2), 207–243 (2008)

    Article  Google Scholar 

  • Zakharov, K., Mitrovic, A., Johnston, L.: Towards emotionally-intelligent pedagogical agents. In: Intelligent Tutoring Systems, 9th Int. Conf., ITS 2008, pp. 19–28. Montreal, Canada (2008)

  • Zhou, X., Conati, C.: Inferring user goals from personality and behavior in a causal model of user affect. In: IUI’03, International Conference on Intelligent User Interfaces, pp. 211–218. Miami, FL (2003)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of British Columbia, Vancouver, BC, V6T 1Z4, Canada

    Cristina Conati

  2. Humanature Studios, Nexon Publishing North America, Vancouver, Canada

    Heather Maclaren

Authors
  1. Cristina Conati
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heather Maclaren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cristina Conati.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Conati, C., Maclaren, H. Empirically building and evaluating a probabilistic model of user affect. User Model User-Adap Inter 19, 267–303 (2009). https://doi.org/10.1007/s11257-009-9062-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-009-9062-8

Keywords

Search

Navigation