Skip to main content
SpringerLink
Log in
Book cover

Toward Robotic Socially Believable Behaving Systems - Volume II pp 93–120Cite as

Fostering User Engagement in Face-to-Face Human-Agent Interactions: A Survey

  • Chapter
  • First Online:

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 106))

Abstract

Embodied conversational agents are capable of carrying a face-to-face interaction with users. Their use is substantially increasing in numerous applications ranging from tutoring systems to ambient assisted living. In such applications, one of the main challenges is to keep the user engaged in the interaction with the agent. The present chapter provides an overview of the scientific issues underlying the engagement paradigm, including a review on methodologies for assessing user engagement in human-agent interaction. It presents three studies that have been conducted within the Greta/VIB platforms. These studies aimed at designing engaging agents using different interaction strategies (alignment and dynamical coupling) and the expression of interpersonal attitudes in multi-party interactions.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    In this case, the task can just be interacting with the agent.

References

  1. Acosta JC, Ward NG (2011) Achieving rapport with turn-by-turn, user-responsive emotional coloring. Speech Commun 53(910):1137–1148

    Article  Google Scholar 

  2. Adelswärd V (1989) Laughter and dialogue: the social significance of laughter in institutional discourse. Nordic J Linguist 12(02):107–136

    Article  Google Scholar 

  3. Albrecht I, Schröder M, Haber J, Seidel H (2005) Mixed feelings: expression of non-basic emotions in a muscle-based talking head. Virtual Real 8(4):201–212

    Article  Google Scholar 

  4. Andre E, Rehm M, Minker W, Bühler D (2004) Endowing spoken language dialogue systems with emotional intelligence. Affective dialogue systems. Springer, Berlin, pp 178–187

    Google Scholar 

  5. Argyle M (1988) Bodily communication, 2nd edn. Methuen, New York

    Google Scholar 

  6. Bazzanella C (2011) Redundancy, repetition, and intensity in discourse. Lang Sci 33(2):243–254

    Article  MathSciNet  Google Scholar 

  7. Bee N, André E, Tober S (2009) Breaking the ice in human-agent communication: eye-gaze based initiation of contact with an embodied conversational agent. In: Proceedings of 9th international conference on Intelligent virtual agents, IVA, Amsterdam, The Netherlands, Sept 14–16, 2009, pp 229–242

    Google Scholar 

  8. Bee N, Pollock C, André E, Walker M (2010) Bossy or wimpy: expressing social dominance by combining gaze and linguistic behaviors. In: Proceedings of the 10th international conference on intelligent virtual agents, Springer, Berlin, Heidelberg, IVA’10, pp 265–271

    Google Scholar 

  9. Bergmann K, Eyssel F, Kopp S (2012) A second chance to make a first impression? how appearance and nonverbal behavior affect perceived warmth and competence of virtual agents over time. In: Nakano Y, Neff M, Paiva A, Walker M (eds) Intelligent virtual agents, Lecture Notes in Computer Science, vol 7502, Springer, Berlin Heidelberg, pp 126–138, doi:10.1007/978-3-642-33197-8_13

    Google Scholar 

  10. Bertrand R, Ferré G, Guardiola M et al (2013) French face-to-face interaction: repetition as a multimodal resource. Interaction, Coverbal Synchrony in Human-Machine, p 141

    Google Scholar 

  11. Beskow J (1997) Animation of talking agents. In: Benoit C, Campbell R (eds) Proceedings of the ESCA workshop on audio-visual speech processing. Rhodes, Greece, pp 149–152

    Google Scholar 

  12. Bickmore T, Giorgino T (2006) Health dialog systems for patients and consumers. J biomed inf 39(5):556–571

    Article  Google Scholar 

  13. Bickmore T, Gruber A, Picard R (2005) Establishing the computer patient working alliance in automated health behavior change interventions. Patient Educ Couns 59(1):21–30. doi:10.1016/j.pec.2004.09.008

    Article  Google Scholar 

  14. Bickmore T, Schulman D, Yin L (2010) Maintaining engagement in long-term interventions with relational agents. Appl Artif Intell 24(6):648–666

    Article  Google Scholar 

  15. Bickmore T, Pfeifer L, Schulman D (2011) Relational agents improve engagement and learning in science museum visitors. Intelligent virtual agents. Springer, Berlin, pp 55–67

    Google Scholar 

  16. Bickmore TW, Picard RW (2005) Establishing and maintaining long-term human-computer relationships. ACM Trans Comput-Hum Interact 12(2):293–327

    Article  Google Scholar 

  17. Bohus D, Horvitz E (2014) Managing human-robot engagement with forecasts and... um... hesitations. In: Proceedings of the 16th international conference on multimodal interaction, ACM, pp 2–9

    Google Scholar 

  18. Boukricha H, Wachsmuth I, Hofstätter A, Grammer K (2009) Pleasure-arousal-dominance driven facial expression simulation. In: Proceedings of third international conference and workshops on Affective computing and intelligent interaction, ACII, Amsterdam, The Netherlands, Sept 10–12 2009, pp 1–7

    Google Scholar 

  19. Branigan HP, Pickering MJ, Cleland AA (2000) Syntactic co-ordination in dialogue. Cognition 75(2):B13–B25

    Article  Google Scholar 

  20. Brennan SE, Clark HH (1996) Conceptual pacts and lexical choice in conversation. J Exp Psychol: Learn, Mem, Cognit 22(6):1482

    Google Scholar 

  21. Bui TD, Heylen D, Poel M, Nijholt A (2001) Generation of facial expressions from emotion using a fuzzy rule based system. In: Stumptner M, Corbett D, Brooks M (eds) Proceedings of 14th Australian joint conference on artificial intelligence (AI 2001). Springer, Adelaide, Australia, pp 83–94

    Google Scholar 

  22. Burgoon J, Buller D, Hale J, de Turck M (1984) Relational messages associated with nonverbal behaviors. Hum Commun Res 10(3):351–378

    Article  Google Scholar 

  23. Buschmeier H, Bergmann K, Kopp S (2009) An alignment-capable microplanner for natural language generation. In: Proceedings of the 12th European workshop on natural language generation, association for computational linguistics, pp 82–89

    Google Scholar 

  24. Cafaro A, Vilhjálmsson HH (2015) First impressions in human-agent virtual encounters. ACM Trans Comput-Hum Interact (TOCHI, forthcoming)

    Google Scholar 

  25. Cafaro A, Vilhjálmsson HH, Bickmore TW, Heylen D, Schulman D (2013) First impressions in user-agent encounters: the impact of an agent’s nonverbal behavior on users’ relational decisions. In: Proceedings of the 2013 international conference on autonomous agents and multi-agent systems, international foundation for autonomous agents and multiagent systems, Richland, SC, AAMAS ’13, pp 1201–1202. http://dl.acm.org/citation.cfm?id=2484920.2485142

  26. Cafaro A, Vilhjlmsson H, Bickmore T, Heylen D, Jhannsdttir K, Valgarsson G (2012) First impressions: users judgments of virtual agents personality and interpersonal attitude in first encounters. In: Nakano Y, Neff M, Paiva A, Walker M (eds) Intelligent virtual agents, Lecture notes in computer science, vol 7502, Springer, Berlin Heidelberg, Richland, SC, pp 67–80. doi:10.1007/978-3-642-33197-8_7

    Google Scholar 

  27. Callejas Z, Griol D (2011) López-Cózar R (2011) Predicting user mental states in spoken dialogue systems. EURASIP J Adv Signal Proc 1:6

    Article  Google Scholar 

  28. Campano S, Sabouret N (2009) A socio-emotional model of impoliteness for non-player characters. In: 3rd international conference on affective computing and intelligent interaction and workshops. ACII 2009. IEEE, pp 1–7

    Google Scholar 

  29. Campano S, Durand J, Clavel C (2014) Comparative analysis of verbal alignment in human-human and human-agent interactions. In: Proceedings of the 9th international conference on language resources and evaluation (LREC’14), European language resources association (ELRA)

    Google Scholar 

  30. Campano S, Langlet C, Glas N, Clavel C, Pelachaud C (2015) An eca expressing appreciations. In: First international workshop on engagement in human computer interaction ENHANCE’15 held in conjuction with the 6th international conference on affective computing and intelligent interaction (ACII 2015), IEEE, sept 2015 (to appear)

    Google Scholar 

  31. Campano S, Clavel C, Pelachaud C (2015) I like this painting too: when an eca shares appreciations to engage users. In: Proceedings of the 14th international joint conference on Autonomous agents and multiagent systems

    Google Scholar 

  32. Cassell J (2000) Embodied conversational agents. MIT press, Cambridge

    Google Scholar 

  33. Castellano G, Mancini M, Peters C, McOwan PW (2012) Expressive copying behavior for social agents: a perceptual analysis. IEEE Trans Syst, Man Cybern, Part A: Syst Hum 42(3):776–783

    Article  Google Scholar 

  34. Chartrand TL, Bargh JA (1999) The chameleon effect: the perception-behavior link and social interaction. J pers soc psychol 76(6):893

    Article  Google Scholar 

  35. Choi A, Melo CD, Woo W, Gratch J (2012) Affective engagement to emotional facial expressions of embodied social agents in a decision-making game. Comput Animat Virtual Worlds 23(3–4):331–342

    Google Scholar 

  36. Chollet M, Ochs M, Clavel C, Pelachaud C, (2013) A multimodal corpus approach to the design of virtual recruiters. In, (2013) humaine association conference on affective computing and intelligent interaction, ACII 2013. Geneva, Switzerland, Sept 2–5:19–24

    Google Scholar 

  37. Clavel C, Callejas Z (2015) Sentiment analysis: from opinion mining to human-agent interaction. IEEE Trans Affect Comput 99:1. doi:10.1109/TAFFC.2015.2444846

    Article  Google Scholar 

  38. Clavel C, Vasilescu I, Devillers L, Richard G, Ehrette T (2008) Fear-type emotions recognition for future audio-based surveillance systems. Speech Commun 50:487–503

    Google Scholar 

  39. Courgeon M, Clavel C, Martin J (2014) Modeling facial signs of appraisal during interaction: impact on users’ perception and behavior. In: International conference on autonomous agents and multi-agent systems, AAMAS ’14, Paris, France, May 5–9 2014, pp 765–772

    Google Scholar 

  40. Creswell JW (2013) Research design: Qualitative, quantitative, and mixed methods approaches (Sage publications)

    Google Scholar 

  41. De Carolis B, Pelachaud C, Poggi I, Steedman M (2004) Apml, a markup language for believable behavior generation. In: Prendinger H, Ishizuka M (eds) Life-like characters, cognitive technologies, Springer, Berlin Heidelberg, pp 65–85. doi:10.1007/978-3-662-08373-4_4

    Google Scholar 

  42. Delaherche E, Chetouani M, Mahdhaoui A, Saint-Georges C, Viaux S, Cohen D (2012) Interpersonal synchrony: a survey of evaluation methods across disciplines. IEEE Trans Affect Comput 3(3):349–365

    Article  Google Scholar 

  43. Dias J, Paiva A (2005) Feeling and reasoning: a computational model for emotional agents. In: Proceedings of 12th Portuguese conference on artificial intelligence, EPIA 2005, Springer, pp 127–140

    Google Scholar 

  44. Ding Y, Prepin K, Huang J, Pelachaud C, Artières T (2014) Laughter animation synthesis. In: Proceedings of the 2014 international conference on autonomous agents and multi-agent systems, AAMAS ’14, pp 773–780

    Google Scholar 

  45. D’Mello S, Graesser A (2013) AutoTutor and affective autotutor: learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Trans Interact Intell Syst 2(4):1–39

    Article  Google Scholar 

  46. D’Mello S, Olney A, Williams C, Hays P (2012) Gaze tutor: a gaze-reactive intelligent tutoring system. Int J hum-comput stud 70(5):377–398

    Article  Google Scholar 

  47. Dybala P, Ptaszynski M, Rzepka R, Araki K (2009) Activating humans with humor - a dialogue system that users want to interact with. IEICE Trans Inf Syst E92-D(12):2394–2401

    Google Scholar 

  48. Ekman P (2003) Emotions revealed. Times Books (US), London, New York. Weidenfeld and Nicolson (world)

    Google Scholar 

  49. Ekman P, Friesen W (1975) Unmasking the face: a guide to recognizing emotions from facial clues. Prentice-Hall Inc, Englewood Cliffs

    Google Scholar 

  50. Ekman P, Friesen W, Hager J (2002) The facial action coding system, 2nd edn. Research Nexus eBook, London, Salt Lake City, Weidenfeld and Nicolson (world)

    Google Scholar 

  51. Fontaine JR, Scherer KR, Roesch EB, Ellsworth P (2007) The world of emotion is not two-dimensional. Psychol Sci 13:1050–1057

    Article  Google Scholar 

  52. Fukayama A, Ohno T, Mukawa N, Sawaki M, Hagita N (2002) Messages embedded in gaze of interface agents–impression management with agent’s gaze. In: Proceedings of the SIGCHI conference on human factors in computing systems, ACM, New York, NY, USA, CHI ’02, pp 41–48. doi:10.1145/503376.503385

  53. Garrod S, Anderson A (1987) Saying what you mean in dialogue: a study in conceptual and semantic co-ordination. Cognition 27(2):181–218

    Article  Google Scholar 

  54. Giles H, Coupland N, Coupland J (1991) Contexts of accommodation: developments in applied sociolinguistics, Cambridge, Cambridge University Press, chap 1–Accommodation theory: communication context, and consequence

    Google Scholar 

  55. Gillies M, Ballin D (2003) A model of interpersonal attitude and posture generation. In: Proceedings of 4th international workshop on Intelligent agents, IVA 2003, Kloster Irsee, Germany, Sept 15–17 2003, pp 88–92

    Google Scholar 

  56. Gillies M, Ballin D (2004) Integrating autonomous behavior and user control for believable agents. In: 3rd International joint conference on autonomous agents and multiagent systems AAMAS, New York, NY, USA, August 19–23 2004, pp 336–343

    Google Scholar 

  57. Glas N, Pelachaud C (2014) Politeness versus perceived engagement: an experimental study. In: Proceedings of the 11th international workshop on natural language processing and cognitive science

    Google Scholar 

  58. Gockley R, Bruce A, Forlizzi J, Michalowski M, Mundell A, Rosenthal S, Sellner B, Simmons R, Snipes K, Schultz A, Wang J (2005) Designing robots for long-term social interaction. In: IEEE/RSJ International conference on intelligent robots and systems (IROS), pp 1338–1343. doi:10.1109/IROS.2005.1545303

  59. Grammer K, Oberzaucher E (2006) The reconstruction of facial expressions in embodied systems. ZiF: mitteilungen, vol 2

    Google Scholar 

  60. Gratch J, Kang SH, Wang N (2013) Using social agents to explore theories of rapport and emotional resonance. Oxford University Press, Oxford, Social emotions in nature and artifact, p 181

    Google Scholar 

  61. Griol D, Molina JM, Callejas Z (2014) Modeling the user state for context-aware spoken interaction in ambient assisted living. Appl Intell 40:1–23

    Article  Google Scholar 

  62. Hall ET (1966) The hidden dimension. Doubleday, Garden City

    Google Scholar 

  63. Hall L, Woods S, Aylett R, Newall L, Paiva A (2005) Achieving empathic engagement through affective interaction with synthetic characters. Affective computing and intelligent interaction. Springer, Heidelberg, pp 731–738

    Google Scholar 

  64. Hess U, Philippot P, Blairy S (1999) The social context of nonverbal behavior mimicry. cambridge University Press, Cambridge, p 213

    Google Scholar 

  65. Heylen D, Kopp S, Marsella SC, Pelachaud C, Vilhjálmsson HH (2008) The next step towards a function markup language. Proceedings of the 8th international conference on Intelligent Virtual Agents. Springer, Berlin, IVA, pp 270–280

    Google Scholar 

  66. Huber T, Ruch W (2007) Laughter as a uniform category? a historic analysis of different types of laughter. In: 10th Congress of the Swiss Society of Psychology. University of Zurich, Switzerland

    Google Scholar 

  67. Ivaldi S, Anzalone SM, Rousseau W, Sigaud O, Chetouani M (2014) Robot initiative in a team learning task increases the rhythm of interaction but not the perceived engagement. Front Neurorobot 8:5

    Article  Google Scholar 

  68. Izard C (1994) Innate and universal facial expressions: evidence from developmental and cross-cultural research. Psychol Bull 115:288–299

    Article  Google Scholar 

  69. Kendon A (1990) Conducting interaction: patterns of behavior in focused encounters (Studies in interactional sociolinguistics). Cambridge University Press, New York

    Google Scholar 

  70. Kopp S, Krenn B, Marsella S, Marshall AN, Pelachaud C, Pirker H, Thórisson KR, Vilhjálmsson HH (2006) Towards a common framework for multimodal generation: the behavior markup language. Proceedings of the 6th international conference on Intelligent virtual agents. Springer, Heidelberg, IVA, pp 205–217

    Google Scholar 

  71. Kopp S (2010) Social resonance and embodied coordination in face-to-face conversation with artificial interlocutors. Speech Commun 52(6):587–597

    Article  Google Scholar 

  72. Kopp S, Gesellensetter L, Krämer NC, Wachsmuth I (2005) A conversational agent as museum guide design and evaluation of a real-world application. In: Panayiotopoulos T, Gratch J, Aylett R, Ballin D, Olivier P, Rist T (eds) Intelligent virtual agents, Lecture notes in computer science, vol 3661, Springer, Berlin, pp 329–343. doi:10.1007/11550617_28

    Google Scholar 

  73. Lambertz K (2011) Back-channelling: The use of yeah and mm to portray engaged listenership. Griffith working papers in pragmatics and intercultural communication. vol 4, pp 11–18

    Google Scholar 

  74. Langlet C, Clavel C (2014) Modelling user’s attitudinal reactions to the agent utterances: focus on the verbal content. In: Proceedings of 5th international workshop on corpora for research on emotion, sentiment & social signals (ES3), Reykjavik, Iceland

    Google Scholar 

  75. Langlet C, Clavel C (2015) Improving social relationships in face-to-face human-agent interactions: when the agent wants to know users likes and dislikes. In: Proceedings of annual meeting of the association for computational linguistic

    Google Scholar 

  76. Lee J, Marsella S (2006) Nonverbal behavior generator for embodied conversational agents. In: Gratch J, Young M, Aylett R, Ballin D, Olivier P (eds), Proceedings 6th international conference intelligent virtual agents (IVA) Marina Del Rey, CA Springer, LNCS, vol 4133, Aug 21–23 2006, pp 243–255

    Google Scholar 

  77. Litman DJ, Forbes-Riley K (2006) Recognizing student emotions and attitudes on the basis of utterances in spoken tutoring dialogues with both human and computer tutors. Speech Commun 48(5):559–590

    Article  Google Scholar 

  78. Lombard M, Weinstein L, Ditton T (2011) Measuring telepresence: the validity of the temple presence inventory (tpi) in a gaming context. In: Proceedings of 2011 Annual conference of the International Society for Presence Research (ISPR), ISPR 2011

    Google Scholar 

  79. Maat M, Heylen D (2009) Turn management or impression management? In: Proceedings of the 9th international conference on intelligent virtual agents, Springer, Heidelberg, IVA, pp 467–473. doi:10.1007/978-3-642-04380-2_51

    Google Scholar 

  80. Maat MT, Truong KP, Heylen D (2010) How turn-taking strategies influence users’ impressions of an agent. In: Allbeck J, Badler N, Bickmore T, Pelachaud C, Safonova A (eds) Intelligent virtual agents, Springer, Heidelberg, Lecture notes in computer science, vol 6356, pp 441–453. doi:10.1007/978-3-642-15892-6_48

    Google Scholar 

  81. Magnusson M (2000) Discovering hidden time patterns in behavior: T-patterns and their detection. Behav Res Meth, Instrum Comput 32(1):93–110

    Article  Google Scholar 

  82. Martin JR, White PR (2005) The language of evaluation. Palgrave Macmillan Basingstoke, New York

    Google Scholar 

  83. Martin JR, White PR (2005) The Language of Evaluation. Appraisal in English, Palgrave Macmillan Basingstoke, New York

    Google Scholar 

  84. Mehrabian A (1980) Basic dimensions for a general psychological theory: implications for personality, social, environmental, and developmental studies. Oelgeschlager, Gunn & Hain, Cambridge

    Google Scholar 

  85. Mol L, Krahmer E, Maes A, Swerts M (2012) Adaptation in gesture: converging hands or converging minds? J Mem Lang 66(1):249–264

    Article  Google Scholar 

  86. Morency LP, de Kok I, Gratch J (2010) A probabilistic multimodal approach for predicting listener backchannels. Auto Agents Multi-Agent Syst 20(1):70–84

    Article  Google Scholar 

  87. Munezero MD, Suero Montero C, Sutinen E, Pajunen J (2014) Are they different? affect, feeling, emotion, sentiment, and opinion detection in text. IEEE Trans Affect Comput 5:101–111

    Article  Google Scholar 

  88. Nakano YI, Ishii R (2010) Estimating user’s engagement from eye-gaze behaviors in human-agent conversations. In: Proceedings of the 15th international conference on intelligent user interfaces, ACM, New York, IUI, pp 139–148. doi:10.1145/1719970.1719990

  89. Nicolle J, Rapp V, Bailly K, Prevost L, Chetouani M (2012) Robust continuous prediction of human emotions using multiscale dynamic cues. In: Proceedings of the 14th ACM international conference on multimodal interaction, pp 501–508

    Google Scholar 

  90. Niederhoffer KG, Pennebaker JW (2002) Linguistic style matching in social interaction. J Lang Soc Psychol 21(4):337–360

    Article  Google Scholar 

  91. Niewiadomski R, Pelachaud C (2007) Model of facial expressions management for an embodied conversational agent. In: Proceedings of 2nd international conference on affective computing and intelligent interaction ACII, Lisbon

    Google Scholar 

  92. Niewiadomski R, Hyniewska SJ, Pelachaud C (2011) Constraint-based model for synthesis of multimodal sequential expressions of emotions. IEEE Trans Affect Comput 2(3):134–146

    Google Scholar 

  93. Niewiadomski R, Demeure V, Pelachaud C (2010) Warmth, competence, believability and virtual agents. In: Allbeck J, Badler N, Bickmore T, Pelachaud C, Safonova A (eds) Intelligent virtual agents, Lecture notes in computer science, vol 6356, Springer, Heidelberg, pp 272–285. doi:10.1007/978-3-642-15892-6_29

    Google Scholar 

  94. Niewiadomski R, Obaid M, Bevacqua E, Looser J, Anh LQ, Pelachaud C (2011b) Cross-media agent platform. In: Proceedings of the 16th international conference on 3D web technology, ACM, New York, Web 3D, pp 11–19. doi:10.1145/2010425.2010428

  95. Ochs M, Prepin K, Pelachaud C (2013) From emotions to interpersonal stances: Multi-level analysis of smiling virtual characters. In: Proceedings of 2013 humaine association conference on affective computing and intelligent interaction (ACII), IEEE, pp 258–263

    Google Scholar 

  96. Osherenko A, Andre E, Vogt T (2009) Affect sensing in speech: Studying fusion of linguistic and acoustic features. In: Proceedings of affective computing and intelligent interaction (ACII)

    Google Scholar 

  97. Paleari M, Lisetti C (2006) Psychologically grounded avatars expressions. In: Proceedings of first workshop on emotion and computing at KI, 29th annual conference on artificial intelligence, Bremen, Germany

    Google Scholar 

  98. Pardo JS (2006) On phonetic convergence during conversational interaction. J Acoust Soc Am 119(4):2382–2393

    Article  Google Scholar 

  99. Pecune F, Cafaro A, Chollet M, Philippe P, Pelachaud C (2014) Suggestions for extending saiba with the vib platform. In: Workshop on architectures and standards for IVAs, held at the ‘14th international conference on intelligent virtual agents (IVA)’, Bielefeld eCollections, pp 16–20

    Google Scholar 

  100. Pecune F, Mancini M, Biancardi B, Varni G, Volpe G, Ding Y, Pelachaud C, Camurri A (2015) Laughing with a virtual agent. In: Proceedings of the 14th international joint conference on Autonomous agents and multiagent systems

    Google Scholar 

  101. Pedica C, Vilhjálmsson HH (2010) Spontaneous avatar behavior for human territoriality. Appl Artif Intell 24(6):575–593. doi:10.1080/08839514.2010.492165

    Google Scholar 

  102. Pekrun R, Cusack A, Murayama K, Elliot AJ, Thomas K (2014) The power of anticipated feedback: effects on students’ achievement goals and achievement emotions. Learn Instruct 29:115–124

    Article  Google Scholar 

  103. Pelachaud C, Badler N, Steedman M (1996) Generating facial expressions for speech. Cognit Sci 20(1):1–46

    Article  Google Scholar 

  104. Perrin L, Deshaies D, Paradis C (2003) Pragmatic functions of local diaphonic repetitions in conversation. J Pragmat 35(12):1843–1860

    Article  Google Scholar 

  105. Peters C, Castellano G, de Freitas S (2009) An exploration of user engagement in hci. In: Proceedings of the international workshop on affective-aware virtual agents and social robots, ACM, p 9

    Google Scholar 

  106. Pickering MJ, Garrod S (2004) Toward a mechanistic psychology of dialogue. Behav brain sci 27(02):169–190

    Google Scholar 

  107. Plutchnik R (1980) Emotion: a psychoevolutionary synthesis. Harper and Row, NY

    Google Scholar 

  108. Poggi I (2007) Mind, hands, face and body: a goal and belief view of multimodal communication. Weidler, Berlin

    Google Scholar 

  109. Prepin K, Ochs M, Pelachaud C (2013) Beyond backchannels: co-construction of dyadic stance by reciprocal reinforcement of smiles between virtual agents. In: International conference CogSci (Annual conference of the cognitive science society)

    Google Scholar 

  110. Provine RR (2001) Laughter: a scientific investigation. Penguin, New York

    Google Scholar 

  111. Ravenet B, Ochs M, Pelachaud C (2013) From a user-created corpus of virtual agents non-verbal behavior to a computational model of interpersonal attitudes. In: Aylett R, Krenn B, Pelachaud C, Shimodaira H (eds) Intelligent virtual agents, Lecture notes in computer science. Springer, Heidelberg, pp 263–274. doi:10.1007/978-3-642-40415-3_23

    Chapter  Google Scholar 

  112. Ravenet B, Ochs M, Pelachaud C (2013) From a user-created corpus of virtual agent’s non-verbal behavior to a computational model of interpersonal attitudes. In: Proceedings 13th international conference intelligent virtual agents, IVA 2013, Edinburgh, UK, August 29–31 2013, pp 263–274

    Google Scholar 

  113. Ravenet B, Cafaro A, Ochs M, Pelachaud C (2014) Interpersonal attitude of a speaking agent in simulated group conversations. In: Bickmore T, Marsella S, Sidner C (eds) Intelligent virtual agents, Lecture notes in computer science, vol 8637, Springer International Publishing, pp 345–349. doi:10.1007/978-3-319-09767-1_45

    Google Scholar 

  114. Ravenet B, Cafaro A, Ochs M, Pelachaud C (2014) Interpersonal attitude of a speaking agent in simulated group conversations. In: Proceedings of 14th international conference on intelligent virtual agents, IVA 2014, Boston, August 27–29, 2014, pp 345–349

    Google Scholar 

  115. Reynolds C (1999) Steering behaviors for autonomous characters. Miller Freeman Game Groups, San Francisco, Proceedings of the game developers conference, p 763

    Google Scholar 

  116. Rich C, Sidner CL (2012) Using collaborative discourse theory to partially automate dialogue tree authoring. In: Proceedings of the 12th international conference on intelligent virtual agents, Springer, Heidelberg, IVA’12, pp 327–340. doi:10.1007/978-3-642-33197-8_34

    Google Scholar 

  117. Russell J (1980) A circumplex model of affect. J Person Soc Psychol 39:1161–1178

    Article  Google Scholar 

  118. Ruttkay Z, Noot H, ten Hagen P (2003) Emotion disc and emotion squares: tools to explore the facial expression face. Comput Graph Forum 22(1):49–53

    Article  Google Scholar 

  119. Scherer K (2000) Emotion. In: Stroebe W, Hewstone M (eds) Introduction to social psychology, a European perspective. Oxford Blackwell Publishers, Oxford, pp 151–191

    Google Scholar 

  120. Scherer K (2005) What are emotions? And how can they be measured? Soc sci inf 44(4):695–729

    Article  Google Scholar 

  121. Schmitt A, Polzehl T, Minker W (2010) Facing reality: simulating deployment of anger recognition in IVR systems. In: Lee GG, Mariani J, Minker W, Nakamura S (eds) Spoken dialogue systems for ambient environments. Lecture notes in computer scienceSpringer, Heidelberg, pp 122–131

    Chapter  Google Scholar 

  122. Schroder M, Bevacqua E, Cowie R, Eyben F, Gunes H, Heylen D, Ter Maat M, McKeown G, Pammi S, Pantic M et al (2012) Building autonomous sensitive artificial listeners. IEEE Trans Affect Comput 3(2):165–183

    Article  Google Scholar 

  123. Schuller B, Rigoll G, Lang M (2004) Speech emotion recognition combining acoustic features and linguistic information in a hybrid support vector machine-belief network architecture. Proceedings of ICASSP, Montreal 1:577–580

    Google Scholar 

  124. Schuller B, Batliner A, Steidl S, Seppi D (2011) Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge. Speech Commun 53(9):1062–1087

    Article  Google Scholar 

  125. Sidner CL, Dzikovska M (2002) Human-robot interaction: engagement between humans and robots for hosting activities. In: Proceedings of the 4th IEEE international conference on multimodal interfaces, IEEE Computer Society, p 123

    Google Scholar 

  126. Sidner CL, Lee C, Kidd CD, Lesh N, Rich C (2005) Explorations in engagement for humans and robots. Artif Intell 166(1):140–164

    Article  Google Scholar 

  127. Silverman LH (1999) Meaning making matters: communication, consequences, and exhibit design. Exhibitionist

    Google Scholar 

  128. Smith C, Crook N, Dobnik S, Charlton D (2011) Interaction strategies for an affective conversational agent. Presence: teleoperators and virtual environments. MIT Press, Cambridge, pp 395–411

    Google Scholar 

  129. Srikant R, Agrawal R (1996) Mining sequential patterns: generalizations and performance improvements. Adv Database Technol 1057:1–17

    Google Scholar 

  130. Stivers T (2008) Stance, alignment, and affiliation during storytelling: when nodding is a token of affiliation. Res Lang soc interact 41(1):31–57

    Article  Google Scholar 

  131. Svennevig J (2004) Other-repetition as display of hearing, understanding and emotional stance. Discourse studies 6(4):489–516

    Article  Google Scholar 

  132. Takashima K, Omori Y, Yoshimoto Y, Itoh Y, Kitamura Y, Kishino F (2008) Effects of avatar’s blinking animation on person impressions. In: Proceedings of graphics interface, Toronto, Ontario, Canada, GI ’08, Canadian Information Processing Society, pp 169–176. doi:10.1145/1375714.1375744

  133. Tannen D (1992) Talking voices: repetition, dialogue, and imagery in conversational discourse. cambridge University Press, Cambridge

    Google Scholar 

  134. Traum D, Aggarwal P, Artstein R, Foutz S, Gerten J, Katsamanis A, Leuski A, Noren D, Swartout W (2012) Ada and grace: Direct interaction with museum visitors. In: Nakano Y, Neff M, Paiva A, Walker M (eds) Intelligent virtual agents, Lecture notes in computer science, vol 7502, Berlin Heidelberg, Springer, pp 245–251. doi:10.1007/978-3-642-33197-8_25

    Google Scholar 

  135. Truong KP, Poppe R, Heylen D (2010) A rule-based backchannel prediction model using pitch and pause information, pp 3058–3061

    Google Scholar 

  136. Tsapatsoulis N, Raouzaiou A, Kollias S, Cowie R, Douglas-Cowie E (2002) Emotion recognition and synthesis based on MPEG-4 FAPs in MPEG-4 facial animation. In: Pandzic IS, Forcheimer R (eds) MPEG4 facial animation–the standard, implementations and applications. Wiley, New York

    Google Scholar 

  137. Vilhjálmsson HH, Cantelmo N, Cassell J, E Chafai N, Kipp M, Kopp S, Mancini M, Marsella S, Marshall AN, Pelachaud C, Ruttkay Z, Thórisson KR, Welbergen H, Werf RJ (2007) The behavior markup language: recent developments and challenges. In: Proceedings of the 7th international conference on intelligent virtual agents, Heidelberg, IVA ’07, Springer, pp 99–111

    Google Scholar 

  138. Vinciarelli A, Pantic M, Bourlard H (2009) Social signal processing: survey of an emerging domain. Image Vis Comput 27(12):1743–1759

    Article  Google Scholar 

  139. Wang N, Johnson WL, Mayer RE, Rizzo P, Shaw E, Collins H (2008) The politeness effect: pedagogical agents and learning outcomes. Int J Hum-Comput Studies 66(2):98–112

    Article  Google Scholar 

  140. Ward A, Litman D (2007) Automatically measuring lexical and acoustic/prosodic convergence in tutorial dialog corpora. In: Proceedings of the SLaTE Workshop on speech and language technology in education

    Google Scholar 

  141. With S, Kaiser S (2011) Sequential patterning of facial actions in the production and perception of emotional expressions. Swiss Journal of Psychology / Schweizerische Zeitschrift fr Psychologie / Revue Suisse de Psychologie 70(4):241–252

    Google Scholar 

  142. Yildirim S, Narayanan S, Potamianos A (2011) Detecting emotional state of a child in a conversational computer game. Comput Speech Lang 25(1):29–44

    Article  Google Scholar 

  143. Zhang L (2009) Exploration of affect sensing from speech and metaphorical text. In: learning by playing. Game-based education system design and development, springer, Berlin, pp 251–262

    Google Scholar 

  144. Zhao R, Papangelis A, Cassell J (2014) Towards a dyadic computational model of rapport management for human-virtual agent interaction. Intelligent virtual agents. Springer, Berlin, pp 514–527

    Google Scholar 

Download references

Acknowledgments

The authors would like to thank the GRETA team for its contributions to the Greta and Vib platforms. This work has been supported by the French collaborative project A1:1, the european project ARIA-VALUSPA, and performed within the Labex SMART (ANR-11-LABX-65) supported by French state funds managed by the ANR within the Investissements d’Avenir programme under reference ANR-11-IDEX-0004-02.

Author information

Authors and Affiliations

  1. Telecom-ParisTech, LTCI, CNRS, Université Paris-Saclay, 46 rue Barrault, 75013, Paris, France

    Chloé Clavel, Angelo Cafaro & Catherine Pelachaud

  2. Lab Object’ive, Object’ive, 96 98 rue de Montreuil, 75011, Paris, France

    Sabrina Campano

Authors
  1. Chloé Clavel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Angelo Cafaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sabrina Campano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Catherine Pelachaud
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chloé Clavel .

Editor information

Editors and Affiliations

  1. Department of Psychology, Seconda Università di Napoli and IIASS, Caserta, Italy

    Anna Esposito

  2. Data Sci. Inst., Faculty of Sci. & Tech., Bournemouth University, Poole, United Kingdom

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Clavel, C., Cafaro, A., Campano, S., Pelachaud, C. (2016). Fostering User Engagement in Face-to-Face Human-Agent Interactions: A Survey. In: Esposito, A., Jain, L. (eds) Toward Robotic Socially Believable Behaving Systems - Volume II . Intelligent Systems Reference Library, vol 106. Springer, Cham. https://doi.org/10.1007/978-3-319-31053-4_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-31053-4_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-31052-7

  • Online ISBN: 978-3-319-31053-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation