Abstract
A smile may communicate different communicative intentions depending on subtle characteristics of the facial expression. In this article, we propose an algorithm to determine the morphological and dynamic characteristics of virtual agent’s smiles of amusement, politeness, and embarrassment. The algorithm has been defined based on a virtual agent’s smiles corpus constructed by users and analyzed with a decision tree classification technique. An evaluation, in different contexts, of the resulting smiles has enabled us to validate the proposed algorithm.
Notes
Note that other elements of the face, such as the gaze, the head movements and the eyebrows, influence how a smile is perceived. However, in the presented work, we focus on the influence of the smile and we do not consider the other elements of the face.
Several other works have explored the impact of a virtual agent’s expressions of emotion on user’s perception (for a detailed review on this subject, see Beale and Creed 2009). In this article, we focus primarily on studies that have compared the user’s perception of different virtual agent’s smiles.
The values of the onset and the offset have been defined to be consistent with the values of the duration of the smile. Moreover, as a first step, discrete variables have been considered. To obtain a more fine-grained description of smiles, continuous variables could be considered.
In more detail, the user’s satisfaction is the same for the three smiles (between 5.2 and 5.5).
References
Ambadar Z, Cohn J, Reed L (2009) All smiles are not created equal: morphology and timing of smiles perceived as amused polite, and embarrassed/nervous. J Nonverbal Behav 17–34:238–252
Beale R, Creed C (2009) Affective interaction: how emotional agents affect users. Int J Hum Comput Stud 67(9):755–776
Breiman L, Friedman J, Olsen R, Stone C (1984) Classification and regression trees. Chapman and Hall, London
Duchenne G [1862 (1990)] The mechanism of human facial expression. Cambridge University Press
Ekman P (1986) Telling lies: clues to deceit in the marketplace politics and marriage. Berkley, NY
Ekman P (2003) Darwin, deception, and facial expression. Ann N Y Acad Sci 1000:205–221
Ekman P, Friesen W (1975) Unmasking the Face. A guide to recognizing emotions from facial clues. Prentice-Hall, Inc, Englewood Cliffs
Ekman P, Friesen W (1982) Felt false, and miserable smiles. J Nonverbal Behav 6:238–252
Ekman P, Friesen W, Hager J (2002) The facial action coding system. Weidenfeld and Nicolson, London
Fernandez-Dols J, Ruiz-Belda J (1995) Are smiles a sign of happiness? gold medal winners at the Olympic games. J Pers Soc Psychol 69:1113–1119
Frank M, Ekman P, Friesen W (1993) Behavioral markers and recognizability of the smile of enjoyment. J Pers Soc Psychol 64:83–93
Grammer K, Oberzaucher E (2006) The reconstruction of facial expressions in embodied systems: new approaches to an old problem. ZIF Mitteilungen 2:14–31
Harrigan JA, O’Connell DM (1996) How do you look when feeling anxious? facial displays of anxiety. Pers Individ Differ 21:205–212
Hess U, Kleck RE (1990) Differentiating emotion elicited and deliberate emotional facial expressions. Eur J Soc Psychol 20(5):369–385
Keltner D (1995) Signs of appeasement: evidence for the distinct displays of embarrassment amusement, and shame. J Pers Soc Psychol 68(3):441–454
Keltner D, Buswell BN (1996) Evidence for the distinctness of embarrassment, shame, and guilt: a study of recalled antecedents and facial expressions of emotion. Cogn Emot 10:155–171
Keltner D, Buswell BN (1997) Embarrassment: its distinct form and appeasement functions. Psychol Bull 122:250–270
Krumhuber EG, Manstead ASR (2009) Can Duchenne smiles be feigned? new evidence on felt and false smiles. Emotion 9(6):807–820
Krumhuber E, Manstead A, Cosker D, Marshall D, Rosin P (2008) Effects of dynamic attributes of smiles in human and synthetic faces: a simulated job interview setting. J Nonverbal Behav 33:1–15
Mehrabian A, Russell JA (1974) An approach to environmental psychology. MIT Press, Cambridge
Niewiadomski R, Pelachaud C (2007) Model of facial expressions management for an embodied conversational agent. In: 2nd international conference on affective computing and intelligent interaction (ACII), Lisbon, Portugal, pp 12–23
Niewiadomski R, Hyniewska S, Pelachaud C (2009) Evaluation of multimodal sequential expressions of emotions in ECA. In: Proceedings of third international conference on affective computing and intelligent interaction (ACII), Amsterdam, Holland, pp 635–641, 200
Ochs M, Niewiadmoski R, Pelachaud C (2010) How a virtual agent should smile? morphological and dynamic characteristics of virtual agent’s smiles, In: Proceedings of the intelligent virtual agent conference (IVA)
Poggi I, Pelachaud C (2000) Emotional meaning and expression in performative faces. In: Affective interactions: towards a new generation of computer interfaces
Rakotomalala R (2005) Tanagra: un logiciel gratuit pour l’enseignement et la recherche. In: Proceedings of EGC (Extraction etGestion des Connaissances), pp 697–702
Rehm M, André E (2005) Catch me if you can exploring lying agents in social settings. In: AAMAS, Academic Press, Inc., pp 937–944
Sabini J, Siepmann M, Stein J, Meyerowitz M (2000) Who is embarrassed by what? Cogn Emot 14:213–240
Tanguy E (2006) Emotions: the art of communication applied to virtual actors. PhD thesis, Department of Computer Science, University of Bath, England
Theonas G, Hobbs D, Rigas D (2008) Employing virtual lecturers’ facial expressions in virtual educational environments. Int J Virtual Real 7:31–44
Wang N, Marsella S, Hawkins T (2008) Individual differences in expressive response: a challenge for ECA design. In: International joint conference on autonomous agents and multiagent systems, pp 1289–1292
Acknowledgments
This research has been partially supported by the European Community Seventh Framework Program (FP7/2007-2013), under grant agreement no. 231287 (SSPNet).
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is part of the Supplement Issue on “Social Signals. From Theory to Applications,” guest-edited by Isabella Poggi, Francesca D’Errico, and Alessandro Vinciarelli.
Rights and permissions
About this article
Cite this article
Ochs, M., Niewiadomski, R., Brunet, P. et al. Smiling virtual agent in social context. Cogn Process 13 (Suppl 2), 519–532 (2012). https://doi.org/10.1007/s10339-011-0424-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10339-011-0424-x