Abstract
Human-to-computer interaction in a variety of applications could benefit if systems could accurately analyze and respond to their users’ affect. Although a great deal of research has been conducted on affect recognition, very little of this work has considered what is the appropriate information to extract in specific situations. Towards understanding how specific applications such as affective tutoring and affective entertainment could benefit, we present two experiments. In the first experiment, we found that students’ facial expressions, together with their body actions, gave little information about their internal emotion per se but they would be useful features for predicting their self-reported “true” mental state. In the second experiment, we found significant differences between the facial expressions and self-reported affective state of viewers watching a movie sequence. Our results suggest that the noisy relationship between observable gestures and underlying affect must be accounted for when designing affective computing applications.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Pantic, M., Sebe, N., Cohn, J.F., Huang, T.: Affective multimodal human-computer interaction. In: Proc. ACM. Int. Conf. Multimedia, pp. 669–676. ACM Press, New York (2005)
Picard, R.W.: Affective Computing. MIT Press, Cambridge (2000)
Prendinger, H., Becker, C., Ishizuka, M.: A study in users’ physiological response to an empathic interface agent. Int. J. of Humanoid Robotics 3(3), 371–391 (2006)
Kapoor, A., Burleson, W., Picard, R.W.: Automatic prediction of frustration. International Journal of Human-Computer Studies (in press, 2007)
Darwin, C.: The Expression of the Emotions in Man and Animals. Oxford University Press, Oxford (1998)
Landis, C.: Studies of emotional reactions: I. A preliminary study of facial expression. Journal of Experimental Psychology 8 (1924)
Ekman, P., Friesen, W.: Constants across cultures in the face and emotion. Journal of Personality and Social Psychology 17(2), 124–129 (1971)
Russell, J.A., Bullock, M.: Fuzzy concepts and the perception of emotion in facial expressions. Social Cognition 4, 309–341 (1986)
Ekman, P., Friesen, W.: Facial Action Coding System (FACS) Manual. Consulting Psychologists Press, Palo Alto, Calif (1978)
Fasel, B., Luettin, J.: Automatic facial expression analysis: A survey. Pattern Recognition 36(1), 259–275 (2003)
Pantic, M., Rothkrantz, L.J.M.: Automatic analysis of facial expressions: The state of the art. IEEE PAMI 22(12), 1424–1445 (2000)
Tian, Y., Kanade, T., Cohn, J.F.: Facial expression analysis. In: Li, S.Z., Jain, A.K. (eds.) Handbook of Face Recognition, Springer, Berlin (2005)
Kim, K.K., Kwak, K.C., Chi, S.Y.: Gesture analysis for human-robot interaction. In: Proc. 8th International Conference on Advanced Communication Technology (ICACT), pp. 1824–1827 (2006)
Dadgostar, F., Ryu, H., Sarrafzadeh, A., Overmyer, S.P.: Making sense of student use of non-verbal cues for intelligent tutoring systems. In: Proc. OZCHI (2005)
Tao, J., Tieniu, T.: Affective computing: A review. In: Proc. 1st International Conference on Affective Computing and Intelligent Interaction, pp. 981–995 (2005)
Ekman, P., Friesen, W.: Pictures of Facial Affect. Consulting Psychologists Press, Palo Alto, Calif (1975)
Matsumoto, D., Ekman, P.: American-Japanese cultural differences in intensity ratings of facial expressions of emotion. Motivation and Emotion 13(2), 143–157 (1989)
Cohn-Kanade AU-Coded Facial Expression Database (CMU Database), http://www.cs.cmu.edu/~face/index2.htm
Sun, Y., Sebe, N., Lew, M.S., Gevers, T.: Authentic emotion detection in real-time video. In: Sebe, N., Lew, M.S., Huang, T.S. (eds.) Computer Vision in Human-Computer Interaction. LNCS, vol. 3058, pp. 94–104. Springer, Heidelberg (2004)
Cowie, E.D., Cowie, R., Schröder, M.: A new emotion database: Considerations, sources and scope. In: Proc. ISCA ITRW on Speech and Emotion, pp. 39–44 (2000)
Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: Analysis of affective physiological state. IEEE PAMI 23(10), 1175–1191 (2001)
Scherer, K.R.: What are emotions? And how can they be measured? Social Science Information 44(4), 695–729 (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Abbasi, A.R., Uno, T., Dailey, M.N., Afzulpurkar, N.V. (2007). Towards Knowledge-Based Affective Interaction: Situational Interpretation of Affect. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds) Affective Computing and Intelligent Interaction. ACII 2007. Lecture Notes in Computer Science, vol 4738. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74889-2_40
Download citation
DOI: https://doi.org/10.1007/978-3-540-74889-2_40
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74888-5
Online ISBN: 978-3-540-74889-2
eBook Packages: Computer ScienceComputer Science (R0)