Abstract
Embodied conversational agents are capable of carrying a face-to-face interaction with users. Their use is substantially increasing in numerous applications ranging from tutoring systems to ambient assisted living. In such applications, one of the main challenges is to keep the user engaged in the interaction with the agent. The present chapter provides an overview of the scientific issues underlying the engagement paradigm, including a review on methodologies for assessing user engagement in human-agent interaction. It presents three studies that have been conducted within the Greta/VIB platforms. These studies aimed at designing engaging agents using different interaction strategies (alignment and dynamical coupling) and the expression of interpersonal attitudes in multi-party interactions.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
In this case, the task can just be interacting with the agent.
References
Acosta JC, Ward NG (2011) Achieving rapport with turn-by-turn, user-responsive emotional coloring. Speech Commun 53(910):1137–1148
Adelswärd V (1989) Laughter and dialogue: the social significance of laughter in institutional discourse. Nordic J Linguist 12(02):107–136
Albrecht I, Schröder M, Haber J, Seidel H (2005) Mixed feelings: expression of non-basic emotions in a muscle-based talking head. Virtual Real 8(4):201–212
Andre E, Rehm M, Minker W, Bühler D (2004) Endowing spoken language dialogue systems with emotional intelligence. Affective dialogue systems. Springer, Berlin, pp 178–187
Argyle M (1988) Bodily communication, 2nd edn. Methuen, New York
Bazzanella C (2011) Redundancy, repetition, and intensity in discourse. Lang Sci 33(2):243–254
Bee N, André E, Tober S (2009) Breaking the ice in human-agent communication: eye-gaze based initiation of contact with an embodied conversational agent. In: Proceedings of 9th international conference on Intelligent virtual agents, IVA, Amsterdam, The Netherlands, Sept 14–16, 2009, pp 229–242
Bee N, Pollock C, André E, Walker M (2010) Bossy or wimpy: expressing social dominance by combining gaze and linguistic behaviors. In: Proceedings of the 10th international conference on intelligent virtual agents, Springer, Berlin, Heidelberg, IVA’10, pp 265–271
Bergmann K, Eyssel F, Kopp S (2012) A second chance to make a first impression? how appearance and nonverbal behavior affect perceived warmth and competence of virtual agents over time. In: Nakano Y, Neff M, Paiva A, Walker M (eds) Intelligent virtual agents, Lecture Notes in Computer Science, vol 7502, Springer, Berlin Heidelberg, pp 126–138, doi:10.1007/978-3-642-33197-8_13
Bertrand R, Ferré G, Guardiola M et al (2013) French face-to-face interaction: repetition as a multimodal resource. Interaction, Coverbal Synchrony in Human-Machine, p 141
Beskow J (1997) Animation of talking agents. In: Benoit C, Campbell R (eds) Proceedings of the ESCA workshop on audio-visual speech processing. Rhodes, Greece, pp 149–152
Bickmore T, Giorgino T (2006) Health dialog systems for patients and consumers. J biomed inf 39(5):556–571
Bickmore T, Gruber A, Picard R (2005) Establishing the computer patient working alliance in automated health behavior change interventions. Patient Educ Couns 59(1):21–30. doi:10.1016/j.pec.2004.09.008
Bickmore T, Schulman D, Yin L (2010) Maintaining engagement in long-term interventions with relational agents. Appl Artif Intell 24(6):648–666
Bickmore T, Pfeifer L, Schulman D (2011) Relational agents improve engagement and learning in science museum visitors. Intelligent virtual agents. Springer, Berlin, pp 55–67
Bickmore TW, Picard RW (2005) Establishing and maintaining long-term human-computer relationships. ACM Trans Comput-Hum Interact 12(2):293–327
Bohus D, Horvitz E (2014) Managing human-robot engagement with forecasts and... um... hesitations. In: Proceedings of the 16th international conference on multimodal interaction, ACM, pp 2–9
Boukricha H, Wachsmuth I, Hofstätter A, Grammer K (2009) Pleasure-arousal-dominance driven facial expression simulation. In: Proceedings of third international conference and workshops on Affective computing and intelligent interaction, ACII, Amsterdam, The Netherlands, Sept 10–12 2009, pp 1–7
Branigan HP, Pickering MJ, Cleland AA (2000) Syntactic co-ordination in dialogue. Cognition 75(2):B13–B25
Brennan SE, Clark HH (1996) Conceptual pacts and lexical choice in conversation. J Exp Psychol: Learn, Mem, Cognit 22(6):1482
Bui TD, Heylen D, Poel M, Nijholt A (2001) Generation of facial expressions from emotion using a fuzzy rule based system. In: Stumptner M, Corbett D, Brooks M (eds) Proceedings of 14th Australian joint conference on artificial intelligence (AI 2001). Springer, Adelaide, Australia, pp 83–94
Burgoon J, Buller D, Hale J, de Turck M (1984) Relational messages associated with nonverbal behaviors. Hum Commun Res 10(3):351–378
Buschmeier H, Bergmann K, Kopp S (2009) An alignment-capable microplanner for natural language generation. In: Proceedings of the 12th European workshop on natural language generation, association for computational linguistics, pp 82–89
Cafaro A, Vilhjálmsson HH (2015) First impressions in human-agent virtual encounters. ACM Trans Comput-Hum Interact (TOCHI, forthcoming)
Cafaro A, Vilhjálmsson HH, Bickmore TW, Heylen D, Schulman D (2013) First impressions in user-agent encounters: the impact of an agent’s nonverbal behavior on users’ relational decisions. In: Proceedings of the 2013 international conference on autonomous agents and multi-agent systems, international foundation for autonomous agents and multiagent systems, Richland, SC, AAMAS ’13, pp 1201–1202. http://dl.acm.org/citation.cfm?id=2484920.2485142
Cafaro A, Vilhjlmsson H, Bickmore T, Heylen D, Jhannsdttir K, Valgarsson G (2012) First impressions: users judgments of virtual agents personality and interpersonal attitude in first encounters. In: Nakano Y, Neff M, Paiva A, Walker M (eds) Intelligent virtual agents, Lecture notes in computer science, vol 7502, Springer, Berlin Heidelberg, Richland, SC, pp 67–80. doi:10.1007/978-3-642-33197-8_7
Callejas Z, Griol D (2011) López-Cózar R (2011) Predicting user mental states in spoken dialogue systems. EURASIP J Adv Signal Proc 1:6
Campano S, Sabouret N (2009) A socio-emotional model of impoliteness for non-player characters. In: 3rd international conference on affective computing and intelligent interaction and workshops. ACII 2009. IEEE, pp 1–7
Campano S, Durand J, Clavel C (2014) Comparative analysis of verbal alignment in human-human and human-agent interactions. In: Proceedings of the 9th international conference on language resources and evaluation (LREC’14), European language resources association (ELRA)
Campano S, Langlet C, Glas N, Clavel C, Pelachaud C (2015) An eca expressing appreciations. In: First international workshop on engagement in human computer interaction ENHANCE’15 held in conjuction with the 6th international conference on affective computing and intelligent interaction (ACII 2015), IEEE, sept 2015 (to appear)
Campano S, Clavel C, Pelachaud C (2015) I like this painting too: when an eca shares appreciations to engage users. In: Proceedings of the 14th international joint conference on Autonomous agents and multiagent systems
Cassell J (2000) Embodied conversational agents. MIT press, Cambridge
Castellano G, Mancini M, Peters C, McOwan PW (2012) Expressive copying behavior for social agents: a perceptual analysis. IEEE Trans Syst, Man Cybern, Part A: Syst Hum 42(3):776–783
Chartrand TL, Bargh JA (1999) The chameleon effect: the perception-behavior link and social interaction. J pers soc psychol 76(6):893
Choi A, Melo CD, Woo W, Gratch J (2012) Affective engagement to emotional facial expressions of embodied social agents in a decision-making game. Comput Animat Virtual Worlds 23(3–4):331–342
Chollet M, Ochs M, Clavel C, Pelachaud C, (2013) A multimodal corpus approach to the design of virtual recruiters. In, (2013) humaine association conference on affective computing and intelligent interaction, ACII 2013. Geneva, Switzerland, Sept 2–5:19–24
Clavel C, Callejas Z (2015) Sentiment analysis: from opinion mining to human-agent interaction. IEEE Trans Affect Comput 99:1. doi:10.1109/TAFFC.2015.2444846
Clavel C, Vasilescu I, Devillers L, Richard G, Ehrette T (2008) Fear-type emotions recognition for future audio-based surveillance systems. Speech Commun 50:487–503
Courgeon M, Clavel C, Martin J (2014) Modeling facial signs of appraisal during interaction: impact on users’ perception and behavior. In: International conference on autonomous agents and multi-agent systems, AAMAS ’14, Paris, France, May 5–9 2014, pp 765–772
Creswell JW (2013) Research design: Qualitative, quantitative, and mixed methods approaches (Sage publications)
De Carolis B, Pelachaud C, Poggi I, Steedman M (2004) Apml, a markup language for believable behavior generation. In: Prendinger H, Ishizuka M (eds) Life-like characters, cognitive technologies, Springer, Berlin Heidelberg, pp 65–85. doi:10.1007/978-3-662-08373-4_4
Delaherche E, Chetouani M, Mahdhaoui A, Saint-Georges C, Viaux S, Cohen D (2012) Interpersonal synchrony: a survey of evaluation methods across disciplines. IEEE Trans Affect Comput 3(3):349–365
Dias J, Paiva A (2005) Feeling and reasoning: a computational model for emotional agents. In: Proceedings of 12th Portuguese conference on artificial intelligence, EPIA 2005, Springer, pp 127–140
Ding Y, Prepin K, Huang J, Pelachaud C, Artières T (2014) Laughter animation synthesis. In: Proceedings of the 2014 international conference on autonomous agents and multi-agent systems, AAMAS ’14, pp 773–780
D’Mello S, Graesser A (2013) AutoTutor and affective autotutor: learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Trans Interact Intell Syst 2(4):1–39
D’Mello S, Olney A, Williams C, Hays P (2012) Gaze tutor: a gaze-reactive intelligent tutoring system. Int J hum-comput stud 70(5):377–398
Dybala P, Ptaszynski M, Rzepka R, Araki K (2009) Activating humans with humor - a dialogue system that users want to interact with. IEICE Trans Inf Syst E92-D(12):2394–2401
Ekman P (2003) Emotions revealed. Times Books (US), London, New York. Weidenfeld and Nicolson (world)
Ekman P, Friesen W (1975) Unmasking the face: a guide to recognizing emotions from facial clues. Prentice-Hall Inc, Englewood Cliffs
Ekman P, Friesen W, Hager J (2002) The facial action coding system, 2nd edn. Research Nexus eBook, London, Salt Lake City, Weidenfeld and Nicolson (world)
Fontaine JR, Scherer KR, Roesch EB, Ellsworth P (2007) The world of emotion is not two-dimensional. Psychol Sci 13:1050–1057
Fukayama A, Ohno T, Mukawa N, Sawaki M, Hagita N (2002) Messages embedded in gaze of interface agents–impression management with agent’s gaze. In: Proceedings of the SIGCHI conference on human factors in computing systems, ACM, New York, NY, USA, CHI ’02, pp 41–48. doi:10.1145/503376.503385
Garrod S, Anderson A (1987) Saying what you mean in dialogue: a study in conceptual and semantic co-ordination. Cognition 27(2):181–218
Giles H, Coupland N, Coupland J (1991) Contexts of accommodation: developments in applied sociolinguistics, Cambridge, Cambridge University Press, chap 1–Accommodation theory: communication context, and consequence
Gillies M, Ballin D (2003) A model of interpersonal attitude and posture generation. In: Proceedings of 4th international workshop on Intelligent agents, IVA 2003, Kloster Irsee, Germany, Sept 15–17 2003, pp 88–92
Gillies M, Ballin D (2004) Integrating autonomous behavior and user control for believable agents. In: 3rd International joint conference on autonomous agents and multiagent systems AAMAS, New York, NY, USA, August 19–23 2004, pp 336–343
Glas N, Pelachaud C (2014) Politeness versus perceived engagement: an experimental study. In: Proceedings of the 11th international workshop on natural language processing and cognitive science
Gockley R, Bruce A, Forlizzi J, Michalowski M, Mundell A, Rosenthal S, Sellner B, Simmons R, Snipes K, Schultz A, Wang J (2005) Designing robots for long-term social interaction. In: IEEE/RSJ International conference on intelligent robots and systems (IROS), pp 1338–1343. doi:10.1109/IROS.2005.1545303
Grammer K, Oberzaucher E (2006) The reconstruction of facial expressions in embodied systems. ZiF: mitteilungen, vol 2
Gratch J, Kang SH, Wang N (2013) Using social agents to explore theories of rapport and emotional resonance. Oxford University Press, Oxford, Social emotions in nature and artifact, p 181
Griol D, Molina JM, Callejas Z (2014) Modeling the user state for context-aware spoken interaction in ambient assisted living. Appl Intell 40:1–23
Hall ET (1966) The hidden dimension. Doubleday, Garden City
Hall L, Woods S, Aylett R, Newall L, Paiva A (2005) Achieving empathic engagement through affective interaction with synthetic characters. Affective computing and intelligent interaction. Springer, Heidelberg, pp 731–738
Hess U, Philippot P, Blairy S (1999) The social context of nonverbal behavior mimicry. cambridge University Press, Cambridge, p 213
Heylen D, Kopp S, Marsella SC, Pelachaud C, Vilhjálmsson HH (2008) The next step towards a function markup language. Proceedings of the 8th international conference on Intelligent Virtual Agents. Springer, Berlin, IVA, pp 270–280
Huber T, Ruch W (2007) Laughter as a uniform category? a historic analysis of different types of laughter. In: 10th Congress of the Swiss Society of Psychology. University of Zurich, Switzerland
Ivaldi S, Anzalone SM, Rousseau W, Sigaud O, Chetouani M (2014) Robot initiative in a team learning task increases the rhythm of interaction but not the perceived engagement. Front Neurorobot 8:5
Izard C (1994) Innate and universal facial expressions: evidence from developmental and cross-cultural research. Psychol Bull 115:288–299
Kendon A (1990) Conducting interaction: patterns of behavior in focused encounters (Studies in interactional sociolinguistics). Cambridge University Press, New York
Kopp S, Krenn B, Marsella S, Marshall AN, Pelachaud C, Pirker H, Thórisson KR, Vilhjálmsson HH (2006) Towards a common framework for multimodal generation: the behavior markup language. Proceedings of the 6th international conference on Intelligent virtual agents. Springer, Heidelberg, IVA, pp 205–217
Kopp S (2010) Social resonance and embodied coordination in face-to-face conversation with artificial interlocutors. Speech Commun 52(6):587–597
Kopp S, Gesellensetter L, Krämer NC, Wachsmuth I (2005) A conversational agent as museum guide design and evaluation of a real-world application. In: Panayiotopoulos T, Gratch J, Aylett R, Ballin D, Olivier P, Rist T (eds) Intelligent virtual agents, Lecture notes in computer science, vol 3661, Springer, Berlin, pp 329–343. doi:10.1007/11550617_28
Lambertz K (2011) Back-channelling: The use of yeah and mm to portray engaged listenership. Griffith working papers in pragmatics and intercultural communication. vol 4, pp 11–18
Langlet C, Clavel C (2014) Modelling user’s attitudinal reactions to the agent utterances: focus on the verbal content. In: Proceedings of 5th international workshop on corpora for research on emotion, sentiment & social signals (ES3), Reykjavik, Iceland
Langlet C, Clavel C (2015) Improving social relationships in face-to-face human-agent interactions: when the agent wants to know users likes and dislikes. In: Proceedings of annual meeting of the association for computational linguistic
Lee J, Marsella S (2006) Nonverbal behavior generator for embodied conversational agents. In: Gratch J, Young M, Aylett R, Ballin D, Olivier P (eds), Proceedings 6th international conference intelligent virtual agents (IVA) Marina Del Rey, CA Springer, LNCS, vol 4133, Aug 21–23 2006, pp 243–255
Litman DJ, Forbes-Riley K (2006) Recognizing student emotions and attitudes on the basis of utterances in spoken tutoring dialogues with both human and computer tutors. Speech Commun 48(5):559–590
Lombard M, Weinstein L, Ditton T (2011) Measuring telepresence: the validity of the temple presence inventory (tpi) in a gaming context. In: Proceedings of 2011 Annual conference of the International Society for Presence Research (ISPR), ISPR 2011
Maat M, Heylen D (2009) Turn management or impression management? In: Proceedings of the 9th international conference on intelligent virtual agents, Springer, Heidelberg, IVA, pp 467–473. doi:10.1007/978-3-642-04380-2_51
Maat MT, Truong KP, Heylen D (2010) How turn-taking strategies influence users’ impressions of an agent. In: Allbeck J, Badler N, Bickmore T, Pelachaud C, Safonova A (eds) Intelligent virtual agents, Springer, Heidelberg, Lecture notes in computer science, vol 6356, pp 441–453. doi:10.1007/978-3-642-15892-6_48
Magnusson M (2000) Discovering hidden time patterns in behavior: T-patterns and their detection. Behav Res Meth, Instrum Comput 32(1):93–110
Martin JR, White PR (2005) The language of evaluation. Palgrave Macmillan Basingstoke, New York
Martin JR, White PR (2005) The Language of Evaluation. Appraisal in English, Palgrave Macmillan Basingstoke, New York
Mehrabian A (1980) Basic dimensions for a general psychological theory: implications for personality, social, environmental, and developmental studies. Oelgeschlager, Gunn & Hain, Cambridge
Mol L, Krahmer E, Maes A, Swerts M (2012) Adaptation in gesture: converging hands or converging minds? J Mem Lang 66(1):249–264
Morency LP, de Kok I, Gratch J (2010) A probabilistic multimodal approach for predicting listener backchannels. Auto Agents Multi-Agent Syst 20(1):70–84
Munezero MD, Suero Montero C, Sutinen E, Pajunen J (2014) Are they different? affect, feeling, emotion, sentiment, and opinion detection in text. IEEE Trans Affect Comput 5:101–111
Nakano YI, Ishii R (2010) Estimating user’s engagement from eye-gaze behaviors in human-agent conversations. In: Proceedings of the 15th international conference on intelligent user interfaces, ACM, New York, IUI, pp 139–148. doi:10.1145/1719970.1719990
Nicolle J, Rapp V, Bailly K, Prevost L, Chetouani M (2012) Robust continuous prediction of human emotions using multiscale dynamic cues. In: Proceedings of the 14th ACM international conference on multimodal interaction, pp 501–508
Niederhoffer KG, Pennebaker JW (2002) Linguistic style matching in social interaction. J Lang Soc Psychol 21(4):337–360
Niewiadomski R, Pelachaud C (2007) Model of facial expressions management for an embodied conversational agent. In: Proceedings of 2nd international conference on affective computing and intelligent interaction ACII, Lisbon
Niewiadomski R, Hyniewska SJ, Pelachaud C (2011) Constraint-based model for synthesis of multimodal sequential expressions of emotions. IEEE Trans Affect Comput 2(3):134–146
Niewiadomski R, Demeure V, Pelachaud C (2010) Warmth, competence, believability and virtual agents. In: Allbeck J, Badler N, Bickmore T, Pelachaud C, Safonova A (eds) Intelligent virtual agents, Lecture notes in computer science, vol 6356, Springer, Heidelberg, pp 272–285. doi:10.1007/978-3-642-15892-6_29
Niewiadomski R, Obaid M, Bevacqua E, Looser J, Anh LQ, Pelachaud C (2011b) Cross-media agent platform. In: Proceedings of the 16th international conference on 3D web technology, ACM, New York, Web 3D, pp 11–19. doi:10.1145/2010425.2010428
Ochs M, Prepin K, Pelachaud C (2013) From emotions to interpersonal stances: Multi-level analysis of smiling virtual characters. In: Proceedings of 2013 humaine association conference on affective computing and intelligent interaction (ACII), IEEE, pp 258–263
Osherenko A, Andre E, Vogt T (2009) Affect sensing in speech: Studying fusion of linguistic and acoustic features. In: Proceedings of affective computing and intelligent interaction (ACII)
Paleari M, Lisetti C (2006) Psychologically grounded avatars expressions. In: Proceedings of first workshop on emotion and computing at KI, 29th annual conference on artificial intelligence, Bremen, Germany
Pardo JS (2006) On phonetic convergence during conversational interaction. J Acoust Soc Am 119(4):2382–2393
Pecune F, Cafaro A, Chollet M, Philippe P, Pelachaud C (2014) Suggestions for extending saiba with the vib platform. In: Workshop on architectures and standards for IVAs, held at the ‘14th international conference on intelligent virtual agents (IVA)’, Bielefeld eCollections, pp 16–20
Pecune F, Mancini M, Biancardi B, Varni G, Volpe G, Ding Y, Pelachaud C, Camurri A (2015) Laughing with a virtual agent. In: Proceedings of the 14th international joint conference on Autonomous agents and multiagent systems
Pedica C, Vilhjálmsson HH (2010) Spontaneous avatar behavior for human territoriality. Appl Artif Intell 24(6):575–593. doi:10.1080/08839514.2010.492165
Pekrun R, Cusack A, Murayama K, Elliot AJ, Thomas K (2014) The power of anticipated feedback: effects on students’ achievement goals and achievement emotions. Learn Instruct 29:115–124
Pelachaud C, Badler N, Steedman M (1996) Generating facial expressions for speech. Cognit Sci 20(1):1–46
Perrin L, Deshaies D, Paradis C (2003) Pragmatic functions of local diaphonic repetitions in conversation. J Pragmat 35(12):1843–1860
Peters C, Castellano G, de Freitas S (2009) An exploration of user engagement in hci. In: Proceedings of the international workshop on affective-aware virtual agents and social robots, ACM, p 9
Pickering MJ, Garrod S (2004) Toward a mechanistic psychology of dialogue. Behav brain sci 27(02):169–190
Plutchnik R (1980) Emotion: a psychoevolutionary synthesis. Harper and Row, NY
Poggi I (2007) Mind, hands, face and body: a goal and belief view of multimodal communication. Weidler, Berlin
Prepin K, Ochs M, Pelachaud C (2013) Beyond backchannels: co-construction of dyadic stance by reciprocal reinforcement of smiles between virtual agents. In: International conference CogSci (Annual conference of the cognitive science society)
Provine RR (2001) Laughter: a scientific investigation. Penguin, New York
Ravenet B, Ochs M, Pelachaud C (2013) From a user-created corpus of virtual agents non-verbal behavior to a computational model of interpersonal attitudes. In: Aylett R, Krenn B, Pelachaud C, Shimodaira H (eds) Intelligent virtual agents, Lecture notes in computer science. Springer, Heidelberg, pp 263–274. doi:10.1007/978-3-642-40415-3_23
Ravenet B, Ochs M, Pelachaud C (2013) From a user-created corpus of virtual agent’s non-verbal behavior to a computational model of interpersonal attitudes. In: Proceedings 13th international conference intelligent virtual agents, IVA 2013, Edinburgh, UK, August 29–31 2013, pp 263–274
Ravenet B, Cafaro A, Ochs M, Pelachaud C (2014) Interpersonal attitude of a speaking agent in simulated group conversations. In: Bickmore T, Marsella S, Sidner C (eds) Intelligent virtual agents, Lecture notes in computer science, vol 8637, Springer International Publishing, pp 345–349. doi:10.1007/978-3-319-09767-1_45
Ravenet B, Cafaro A, Ochs M, Pelachaud C (2014) Interpersonal attitude of a speaking agent in simulated group conversations. In: Proceedings of 14th international conference on intelligent virtual agents, IVA 2014, Boston, August 27–29, 2014, pp 345–349
Reynolds C (1999) Steering behaviors for autonomous characters. Miller Freeman Game Groups, San Francisco, Proceedings of the game developers conference, p 763
Rich C, Sidner CL (2012) Using collaborative discourse theory to partially automate dialogue tree authoring. In: Proceedings of the 12th international conference on intelligent virtual agents, Springer, Heidelberg, IVA’12, pp 327–340. doi:10.1007/978-3-642-33197-8_34
Russell J (1980) A circumplex model of affect. J Person Soc Psychol 39:1161–1178
Ruttkay Z, Noot H, ten Hagen P (2003) Emotion disc and emotion squares: tools to explore the facial expression face. Comput Graph Forum 22(1):49–53
Scherer K (2000) Emotion. In: Stroebe W, Hewstone M (eds) Introduction to social psychology, a European perspective. Oxford Blackwell Publishers, Oxford, pp 151–191
Scherer K (2005) What are emotions? And how can they be measured? Soc sci inf 44(4):695–729
Schmitt A, Polzehl T, Minker W (2010) Facing reality: simulating deployment of anger recognition in IVR systems. In: Lee GG, Mariani J, Minker W, Nakamura S (eds) Spoken dialogue systems for ambient environments. Lecture notes in computer scienceSpringer, Heidelberg, pp 122–131
Schroder M, Bevacqua E, Cowie R, Eyben F, Gunes H, Heylen D, Ter Maat M, McKeown G, Pammi S, Pantic M et al (2012) Building autonomous sensitive artificial listeners. IEEE Trans Affect Comput 3(2):165–183
Schuller B, Rigoll G, Lang M (2004) Speech emotion recognition combining acoustic features and linguistic information in a hybrid support vector machine-belief network architecture. Proceedings of ICASSP, Montreal 1:577–580
Schuller B, Batliner A, Steidl S, Seppi D (2011) Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge. Speech Commun 53(9):1062–1087
Sidner CL, Dzikovska M (2002) Human-robot interaction: engagement between humans and robots for hosting activities. In: Proceedings of the 4th IEEE international conference on multimodal interfaces, IEEE Computer Society, p 123
Sidner CL, Lee C, Kidd CD, Lesh N, Rich C (2005) Explorations in engagement for humans and robots. Artif Intell 166(1):140–164
Silverman LH (1999) Meaning making matters: communication, consequences, and exhibit design. Exhibitionist
Smith C, Crook N, Dobnik S, Charlton D (2011) Interaction strategies for an affective conversational agent. Presence: teleoperators and virtual environments. MIT Press, Cambridge, pp 395–411
Srikant R, Agrawal R (1996) Mining sequential patterns: generalizations and performance improvements. Adv Database Technol 1057:1–17
Stivers T (2008) Stance, alignment, and affiliation during storytelling: when nodding is a token of affiliation. Res Lang soc interact 41(1):31–57
Svennevig J (2004) Other-repetition as display of hearing, understanding and emotional stance. Discourse studies 6(4):489–516
Takashima K, Omori Y, Yoshimoto Y, Itoh Y, Kitamura Y, Kishino F (2008) Effects of avatar’s blinking animation on person impressions. In: Proceedings of graphics interface, Toronto, Ontario, Canada, GI ’08, Canadian Information Processing Society, pp 169–176. doi:10.1145/1375714.1375744
Tannen D (1992) Talking voices: repetition, dialogue, and imagery in conversational discourse. cambridge University Press, Cambridge
Traum D, Aggarwal P, Artstein R, Foutz S, Gerten J, Katsamanis A, Leuski A, Noren D, Swartout W (2012) Ada and grace: Direct interaction with museum visitors. In: Nakano Y, Neff M, Paiva A, Walker M (eds) Intelligent virtual agents, Lecture notes in computer science, vol 7502, Berlin Heidelberg, Springer, pp 245–251. doi:10.1007/978-3-642-33197-8_25
Truong KP, Poppe R, Heylen D (2010) A rule-based backchannel prediction model using pitch and pause information, pp 3058–3061
Tsapatsoulis N, Raouzaiou A, Kollias S, Cowie R, Douglas-Cowie E (2002) Emotion recognition and synthesis based on MPEG-4 FAPs in MPEG-4 facial animation. In: Pandzic IS, Forcheimer R (eds) MPEG4 facial animation–the standard, implementations and applications. Wiley, New York
Vilhjálmsson HH, Cantelmo N, Cassell J, E Chafai N, Kipp M, Kopp S, Mancini M, Marsella S, Marshall AN, Pelachaud C, Ruttkay Z, Thórisson KR, Welbergen H, Werf RJ (2007) The behavior markup language: recent developments and challenges. In: Proceedings of the 7th international conference on intelligent virtual agents, Heidelberg, IVA ’07, Springer, pp 99–111
Vinciarelli A, Pantic M, Bourlard H (2009) Social signal processing: survey of an emerging domain. Image Vis Comput 27(12):1743–1759
Wang N, Johnson WL, Mayer RE, Rizzo P, Shaw E, Collins H (2008) The politeness effect: pedagogical agents and learning outcomes. Int J Hum-Comput Studies 66(2):98–112
Ward A, Litman D (2007) Automatically measuring lexical and acoustic/prosodic convergence in tutorial dialog corpora. In: Proceedings of the SLaTE Workshop on speech and language technology in education
With S, Kaiser S (2011) Sequential patterning of facial actions in the production and perception of emotional expressions. Swiss Journal of Psychology / Schweizerische Zeitschrift fr Psychologie / Revue Suisse de Psychologie 70(4):241–252
Yildirim S, Narayanan S, Potamianos A (2011) Detecting emotional state of a child in a conversational computer game. Comput Speech Lang 25(1):29–44
Zhang L (2009) Exploration of affect sensing from speech and metaphorical text. In: learning by playing. Game-based education system design and development, springer, Berlin, pp 251–262
Zhao R, Papangelis A, Cassell J (2014) Towards a dyadic computational model of rapport management for human-virtual agent interaction. Intelligent virtual agents. Springer, Berlin, pp 514–527
Acknowledgments
The authors would like to thank the GRETA team for its contributions to the Greta and Vib platforms. This work has been supported by the French collaborative project A1:1, the european project ARIA-VALUSPA, and performed within the Labex SMART (ANR-11-LABX-65) supported by French state funds managed by the ANR within the Investissements d’Avenir programme under reference ANR-11-IDEX-0004-02.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Clavel, C., Cafaro, A., Campano, S., Pelachaud, C. (2016). Fostering User Engagement in Face-to-Face Human-Agent Interactions: A Survey. In: Esposito, A., Jain, L. (eds) Toward Robotic Socially Believable Behaving Systems - Volume II . Intelligent Systems Reference Library, vol 106. Springer, Cham. https://doi.org/10.1007/978-3-319-31053-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-31053-4_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-31052-7
Online ISBN: 978-3-319-31053-4
eBook Packages: EngineeringEngineering (R0)