Abstract
Emotion influences the choice of facial expression. In a dialogue the emotional state is co-determined by the events that happen during a dialogue. To enable rich, human like expressivity of a dialogue agent, the facial displays should show a correct expression of the state of the agent in the dialogue. This paper reports about our study in building knowledge on how to appropriately express emotions in face to face communication. We have analyzed the appearance of facial expressions and corresponding dialogue-text (in balloons) of characters of selected cartoon illustrations. From the facial expressions and dialogue-text, we have extracted independently the emotional state and the communicative function. We also collected emotion words from the dialogue-text. The emotional states (label) and the emotion words are represented along two dimensions “arousal” and “valence”. Here, the relationship between facial expressions and text were explored. The final goal of this research is to develop emotional-display rules for a text-based dialogue agent.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
De Carolis, B., Pelachaud, C., Poggi, I., De Rosis, F.: Behavior Planning for a Reflexive Agent. In: Proc. of IJCAI 2001, Portugal, pp. 1059–1066 (2001)
Cassell, J., Vilhjálmsson, H., Bickmore, T.: BEAT – The Behavior Expression Animation Toolkit. In: Proc of SIGGRAPH 2001, Los Angeles, CA, pp. 477–486 (2001)
Condon, W., Osgton, W.: Speech and Body Motion Synchrony of the Speaker-Hearer. In: Horton, D., Jenkins, J. (eds.) The perception of Language, pp. 150–184. Academic Press, London (1971)
Desmet, P.: Designing Emotion, Doctoral Dissertation, TU Delft (2002)
Ekman, P.: Telling Lies: Clues to Deceit in the Marketplace, Marriage, and Politics. WW Norton, New York (2001)
Ekman, P.: Basic Emotions. In: Dalgleish, T., Power, M. (eds.) Handbook of Cognition and Emotion, UK. John Wiley and Sons Ltd, Chichester (1999)
Ekman, P.: About brows: Emotional and Conversational Signals. In: Human Ethology: Claims and limits of a new discipline: contributions to the Colloquium, Cambridge, pp. 169–248 (1979)
Fitrianie, S., Rothkrantz, L.J.M.: An Automated Text-Based Synthetic Face with Emotions for Web Lectures. CCAI 23(1–4), 89–98 (2007)
Fitrianie, S., Rothkrantz, L.J.M.: An Automated Crisis Online Dispatcher. Int. Journal of Emergency Management, Research Method (to appear, 2008)
Fitrianie, S., Wiggers, P., Rothkrantz, L.J.M.: A Multimodal Eliza Using Natural Language Processing and Emotion Recognition. In: Matoušek, V., Mautner, P. (eds.) TSD 2003. LNCS (LNAI), vol. 2807, pp. 394–399. Springer, Heidelberg (2003)
Kiesler, S., Sproull, L.: Social Human Computer Interaction, Human Values and Design of Computer Technology, pp. 191–199. CSLI Pub., Standford (1997)
Liu, H., Lieberman, H., Selker, T.: A Model of Textual Affect Sensing using Real World Knowledge, Technical Report, MIT Media Laboratory, Cambridge (2003)
Mulder, M., Nijholt, A., den Uyl, M., Terpstra, P.: A Lexical Grammatical Implementation of Affect. In: Sojka, P., Kopeček, I., Pala, K. (eds.) TSD 2004. LNCS (LNAI), vol. 3206, pp. 171–178. Springer, Heidelberg (2004)
Niewiadomski, R., Pelachaud, C.: Model of Facial Expressions Management for an Embodied Conversational Agent. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 12–23. Springer, Heidelberg (2007)
Pang, B., Lee, L.: A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts. In: ACL 2004, Spain, pp. 271–278 (2004)
Predinger, H., Ishizuka, M.: Social Role Awareness in Animated Agents. In: Proc. of AGENT 2001, pp. 270–276. ACM, New York (2001)
Schiano, D.J., Erlich, S.M., Rahardja, K., Sheridan, K.: Face to Interface: Facial Affect in (Hu)man and Machine. In: Proc. of ACM CHI 2000, pp. 193–200 (2000)
Tokuhisa, M., Murakami, J., Ikehara, S.: Construction and Evaluation of Text-Dialog Corpus with Emotion Tags Focusing on Facial Expression in Comic. In: Gabrys, B., Howlett, R.J., Jain, L.C. (eds.) KES 2006. LNCS (LNAI), vol. 4253, pp. 715–724. Springer, Heidelberg (2006)
van Vark, R.J., Vreught, J.P., Rothkrantz, L.J.M.: Classification of Public Transport Information Dialogues Using an Information-Based Coding Scheme. In: Dialogue Processing in Spoken Language Systems Workshop, pp. 55–69. Springer, Heidelberg (1997)
Wallace, R.: Alicebot (1995), http://www.Alicebot.org
Weizenbaum, J.: ELIZA – A Computer Program for the Study of Natural Language Communication between Man and Machine. Comm. of ACM 9(1), 36–45 (1966)
Whissell, C.M.: The Dictionary of Affect and Language. In: Plutchik, R., Kellerman, H. (eds.) Emotion: Theory, Research, and Experience, vol. 4, pp. 113–131 (1989)
Wojdel, A., Rothkrantz, L.J.M.: Parametric Generation of Facial Expressions Based on Facs. Computer Graphics forum 4(24), 1–15 (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fitrianie, S., Rothkrantz, L.J.M. (2008). The Generation of Emotional Expressions for a Text-Based Dialogue Agent. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds) Text, Speech and Dialogue. TSD 2008. Lecture Notes in Computer Science(), vol 5246. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87391-4_72
Download citation
DOI: https://doi.org/10.1007/978-3-540-87391-4_72
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87390-7
Online ISBN: 978-3-540-87391-4
eBook Packages: Computer ScienceComputer Science (R0)