Skip to main content

Advertisement

Log in

Smiling virtual agent in social context

Cognitive Processing Aims and scope Submit manuscript

Abstract

A smile may communicate different communicative intentions depending on subtle characteristics of the facial expression. In this article, we propose an algorithm to determine the morphological and dynamic characteristics of virtual agent’s smiles of amusement, politeness, and embarrassment. The algorithm has been defined based on a virtual agent’s smiles corpus constructed by users and analyzed with a decision tree classification technique. An evaluation, in different contexts, of the resulting smiles has enabled us to validate the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Notes

  1. Note that other elements of the face, such as the gaze, the head movements and the eyebrows, influence how a smile is perceived. However, in the presented work, we focus on the influence of the smile and we do not consider the other elements of the face.

  2. Several other works have explored the impact of a virtual agent’s expressions of emotion on user’s perception (for a detailed review on this subject, see Beale and Creed 2009). In this article, we focus primarily on studies that have compared the user’s perception of different virtual agent’s smiles.

  3. The values of the onset and the offset have been defined to be consistent with the values of the duration of the smile. Moreover, as a first step, discrete variables have been considered. To obtain a more fine-grained description of smiles, continuous variables could be considered.

  4. In more detail, the user’s satisfaction is the same for the three smiles (between 5.2 and 5.5).

References

  • Ambadar Z, Cohn J, Reed L (2009) All smiles are not created equal: morphology and timing of smiles perceived as amused polite, and embarrassed/nervous. J Nonverbal Behav 17–34:238–252

    Google Scholar 

  • Beale R, Creed C (2009) Affective interaction: how emotional agents affect users. Int J Hum Comput Stud 67(9):755–776

    Article  Google Scholar 

  • Breiman L, Friedman J, Olsen R, Stone C (1984) Classification and regression trees. Chapman and Hall, London

    Google Scholar 

  • Duchenne G [1862 (1990)] The mechanism of human facial expression. Cambridge University Press

  • Ekman P (1986) Telling lies: clues to deceit in the marketplace politics and marriage. Berkley, NY

    Google Scholar 

  • Ekman P (2003) Darwin, deception, and facial expression. Ann N Y Acad Sci 1000:205–221

    Article  PubMed  Google Scholar 

  • Ekman P, Friesen W (1975) Unmasking the Face. A guide to recognizing emotions from facial clues. Prentice-Hall, Inc, Englewood Cliffs

    Google Scholar 

  • Ekman P, Friesen W (1982) Felt false, and miserable smiles. J Nonverbal Behav 6:238–252

    Article  Google Scholar 

  • Ekman P, Friesen W, Hager J (2002) The facial action coding system. Weidenfeld and Nicolson, London

    Google Scholar 

  • Fernandez-Dols J, Ruiz-Belda J (1995) Are smiles a sign of happiness? gold medal winners at the Olympic games. J Pers Soc Psychol 69:1113–1119

    Article  Google Scholar 

  • Frank M, Ekman P, Friesen W (1993) Behavioral markers and recognizability of the smile of enjoyment. J Pers Soc Psychol 64:83–93

    Article  PubMed  CAS  Google Scholar 

  • Grammer K, Oberzaucher E (2006) The reconstruction of facial expressions in embodied systems: new approaches to an old problem. ZIF Mitteilungen 2:14–31

    Google Scholar 

  • Harrigan JA, O’Connell DM (1996) How do you look when feeling anxious? facial displays of anxiety. Pers Individ Differ 21:205–212

    Article  Google Scholar 

  • Hess U, Kleck RE (1990) Differentiating emotion elicited and deliberate emotional facial expressions. Eur J Soc Psychol 20(5):369–385

    Article  Google Scholar 

  • Keltner D (1995) Signs of appeasement: evidence for the distinct displays of embarrassment amusement, and shame. J Pers Soc Psychol 68(3):441–454

    Article  Google Scholar 

  • Keltner D, Buswell BN (1996) Evidence for the distinctness of embarrassment, shame, and guilt: a study of recalled antecedents and facial expressions of emotion. Cogn Emot 10:155–171

    Article  Google Scholar 

  • Keltner D, Buswell BN (1997) Embarrassment: its distinct form and appeasement functions. Psychol Bull 122:250–270

    Article  PubMed  CAS  Google Scholar 

  • Krumhuber EG, Manstead ASR (2009) Can Duchenne smiles be feigned? new evidence on felt and false smiles. Emotion 9(6):807–820

    Article  PubMed  Google Scholar 

  • Krumhuber E, Manstead A, Cosker D, Marshall D, Rosin P (2008) Effects of dynamic attributes of smiles in human and synthetic faces: a simulated job interview setting. J Nonverbal Behav 33:1–15

    Article  Google Scholar 

  • Mehrabian A, Russell JA (1974) An approach to environmental psychology. MIT Press, Cambridge

    Google Scholar 

  • Niewiadomski R, Pelachaud C (2007) Model of facial expressions management for an embodied conversational agent. In: 2nd international conference on affective computing and intelligent interaction (ACII), Lisbon, Portugal, pp 12–23

  • Niewiadomski R, Hyniewska S, Pelachaud C (2009) Evaluation of multimodal sequential expressions of emotions in ECA. In: Proceedings of third international conference on affective computing and intelligent interaction (ACII), Amsterdam, Holland, pp 635–641, 200

  • Ochs M, Niewiadmoski R, Pelachaud C (2010) How a virtual agent should smile? morphological and dynamic characteristics of virtual agent’s smiles, In: Proceedings of the intelligent virtual agent conference (IVA)

  • Poggi I, Pelachaud C (2000) Emotional meaning and expression in performative faces. In: Affective interactions: towards a new generation of computer interfaces

  • Rakotomalala R (2005) Tanagra: un logiciel gratuit pour l’enseignement et la recherche. In: Proceedings of EGC (Extraction etGestion des Connaissances), pp 697–702

  • Rehm M, André E (2005) Catch me if you can exploring lying agents in social settings. In: AAMAS, Academic Press, Inc., pp 937–944

  • Sabini J, Siepmann M, Stein J, Meyerowitz M (2000) Who is embarrassed by what? Cogn Emot 14:213–240

    Article  Google Scholar 

  • Tanguy E (2006) Emotions: the art of communication applied to virtual actors. PhD thesis, Department of Computer Science, University of Bath, England

  • Theonas G, Hobbs D, Rigas D (2008) Employing virtual lecturers’ facial expressions in virtual educational environments. Int J Virtual Real 7:31–44

    Google Scholar 

  • Wang N, Marsella S, Hawkins T (2008) Individual differences in expressive response: a challenge for ECA design. In: International joint conference on autonomous agents and multiagent systems, pp 1289–1292

Download references

Acknowledgments

This research has been partially supported by the European Community Seventh Framework Program (FP7/2007-2013), under grant agreement no. 231287 (SSPNet).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Magalie Ochs.

Additional information

This article is part of the Supplement Issue on “Social Signals. From Theory to Applications,” guest-edited by Isabella Poggi, Francesca D’Errico, and Alessandro Vinciarelli.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ochs, M., Niewiadomski, R., Brunet, P. et al. Smiling virtual agent in social context. Cogn Process 13 (Suppl 2), 519–532 (2012). https://doi.org/10.1007/s10339-011-0424-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10339-011-0424-x

Keywords

Navigation