Article

Basic components of a face-to-face interaction with a conversational agent: mutual attention and deixis

Authors:

Gérard Bailly,

Frédéric EliseiAuthors Info & Claims

sOc-EUSAI '05: Proceedings of the 2005 joint conference on Smart objects and ambient intelligence: innovative context-aware services: usages and technologies

Pages 247 - 252

https://doi.org/10.1145/1107548.1107610

Published: 12 October 2005 Publication History

Abstract

We present a series of experiments that involve a face-to-face interaction between an embodied conversational agent (ECA) and a human interlocutor. The main challenge is to provide the interlocutor with implicit and explicit signs of mutual interest and attention and of the awareness of environmental conditions in which the interaction takes place. A video realistic talking head with independent head and eye movements was used as a talking agent interacting with a user during a simple card game offering different levels of help and guidance. We analyzed the user performance and how the quality of assistance given by the embodied conversational agent was perceived. The experiment showed that users can profit from its presence and its facial deictic cues.

References

[1]

Bailly, G., Bérar, M., Elisei, F., and Odisio, M. (2003) Audiovisual speech synthesis. International Journal of Speech Technology, 6: p.331--346.

[2]

Baron-Cohen, S., Leslie, A., and Frith, U. (1985) Does the autistic child have a "theory of mind"?, Cognition, 21: p.37--46.

[3]

Breazeal, C. (2002) Designing Sociable Robots,.: The MIT Press.

Digital Library

[4]

Brooks, R. A., Breazeal, C., Marjanovic, M., Scassellati, B., and Williamson, M. (1999) The Cog Project: Building a Humanoid Robot" in Computation for Metaphors, Analogy, and Agents, in Lecture Notes in Artificial Intelligence, C. Nehaniv, Editor. Springer: New York. p. 52--87.

Digital Library

[5]

Cassell, J., Sullivan, J., Prevost, S., and Churchill, E. (2000) Embodied Conversational Agents. Cambridge: MIT Press.

[6]

Driver, J., Davis, G., Riccardelli, P., Kidd, P., Maxwell. E., and Baron-Cohen, S. (1999) Shared attention and the social brain: gaze perception triggers automatic visuospatial orienting in adults. Visual Cognition, 6(5): p.509--540.

[7]

Garau, M., Slater, M., Bee, S., and Sasse, M.-A. (2001) The impact of eye gaze on communication using humanoid avatars. in CHI'01: ACM Conference on Human Factors in Computing Systems. Seattle, WA. p.309--316.

Digital Library

[8]

Langton, S. and Bruce, V. (1999) Reflexive visual orienting in response to the social attention of others. Visual Cognition, 6(5): p.541--567.

[9]

Langton. S., Watt, J., and Bruce, V. (2000) Do the eyes have it? Cues to the direction of social attention. Trends in Cognitive Sciences, 4(2): p.50--59.

[10]

Leslie, A. M. (1994) ToMM, ToBY, and Agency: Core architecture and domain specificity, in Mapping the Mind: Domain specificity in cognition and culture, L. A. Hirschfeld and S. A. Gelman, Editors. Cambridge University Press: Cambridge. p. 119--148.

[11]

Nagao, K. and Takeuchi, A. (1994) Speech dialogue with facial displays: multimodal human-computer conversation. in 32nd conference of the Association for Computational Linguistics. Las Cruces, New Mexico. p. 102--109.

Digital Library

[12]

Posner, M. and Peterson, S. (1990) The attention system of the human brain. Annual Review of Neuroscience, 13: p.25--42.

[13]

Posner, M. I. (1980) Orienting of attention. Quarterly Journal of Experimental Psychology, 32: p.3--25.

[14]

Premack, D. and Woodruff. G. (1978) Does the chimpanzee have a theory of mind?, Behavioral and brain sciences, 1: p.515--526.

[15]

Revéret, L., Bailly, G., and Badin. P. (2000) MOTHER: a new generation of talking heads providing a flexible articulatory control for video-realistic speech animation. in International Conference on Speech and Language Processing. Beijing - China. p.755--758.

[16]

Roccella, S., Carrozza. M. C., Cappiello, G., Dario, P., Cabibihan. J.-J., Zecca. M., Miwa, H., Itoh, K., Matsumoto, M., and Takanishi, A. (2004) Design, fabrication and preliminary results of a novel anthropomorphic hand for humanoid robotics: RCH-1,. in IEEE/RSJ International Conference on Intelligent Robot and Systems. Sendai, Japan. p.266--271.

[17]

Scassellati, B. (2001) Foundations for a theory of mind for a humanoid robot, in Department of Computer Science and Electrical Engineering. MIT: Boston - MA. p. 174.

Digital Library

[18]

Simons, D. J. and Chabris. C. F. (1999) Gorillas in our midst: sustained inattentional blindness for dynamic events. Perception, 28: p. 1059--1074.

[19]

Vatikiotis-Bateson, E., Eigsti, I.-M., Yano, S., and Munhall, K. G. (1998) Eye movement of perceivers during audiovisual speech perception. Perception & Psychophysics, 60: p.926--940.

[20]

Vertegaal, R., Slagter, R., Veer, G.v.d., and Nijholt, A. (2001) Eye gaze patterns in conversations: There is more to conversational agents than meets the eyes. in Conference on Human Factors in Computing Systems. Seattle, USA. p.301--308.

Digital Library

[21]

Yarbus. A. L. (1967) Eye movements during perception of complex objects, in Eye Movements and Vision', L. A. Riggs, Editor. Plenum Press: New York. p. 171--196.

Cited By

Loke MTang KChua GTan YFarbiz FNakajima MWu EMiyata KThalmann DHuang Z(2009)The effect of facial animation on a dancing characterProceedings of the 8th International Conference on Virtual Reality Continuum and its Applications in Industry10.1145/1670252.1670276(107-112)Online publication date: 14-Dec-2009
https://dl.acm.org/doi/10.1145/1670252.1670276
Trafton JBugajska MFransen BRatwani RFong TDautenhahn KScheutz MDemiris Y(2008)Integrating vision and audition within a cognitive architecture to track conversationsProceedings of the 3rd ACM/IEEE international conference on Human robot interaction10.1145/1349822.1349849(201-208)Online publication date: 12-Mar-2008
https://dl.acm.org/doi/10.1145/1349822.1349849

Basic components of a face-to-face interaction with a conversational agent: mutual attention and deixis
1. Human-centered computing

Recommendations

Gaze, conversational agents and face-to-face communication

In this paper, we describe two series of experiments that examine audiovisual face-to-face interaction between naive human viewers and either a human interlocutor or a virtual conversational agent. The main objective is to analyze the interplay between ...
Integrating embodied conversational agent components with a generic framework
Innovations in intelligent agent technology

Embodied Conversational Agents (ECAs) are computer generated life-like characters that interact with human users in face-to-face conversations. To achieve natural multi-modal conversations, ECA systems are sophisticated and require numbers of building ...
Speech and face-to-face communication - An introduction

This issue focuses on face-to-face speech communication. Research works have demonstrated that this communicative situation is essential to language acquisition and development (e.g. naming). Face-to-face communication is in fact much more than speaking ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

sOc-EUSAI '05: Proceedings of the 2005 joint conference on Smart objects and ambient intelligence: innovative context-aware services: usages and technologies

October 2005

316 pages

ISBN:1595933042

DOI:10.1145/1107548

Conference Chairs:
Gerard Bailly
CNRS, France
,
James L. Crowley
INPG - France

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

sOc-EUSAI05

sOc-EUSAI05: Smart Objects & Ambient Intelligence

October 12 - 14, 2005

Grenoble, France

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
288
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)1

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Loke MTang KChua GTan YFarbiz FNakajima MWu EMiyata KThalmann DHuang Z(2009)The effect of facial animation on a dancing characterProceedings of the 8th International Conference on Virtual Reality Continuum and its Applications in Industry10.1145/1670252.1670276(107-112)Online publication date: 14-Dec-2009
https://dl.acm.org/doi/10.1145/1670252.1670276
Trafton JBugajska MFransen BRatwani RFong TDautenhahn KScheutz MDemiris Y(2008)Integrating vision and audition within a cognitive architecture to track conversationsProceedings of the 3rd ACM/IEEE international conference on Human robot interaction10.1145/1349822.1349849(201-208)Online publication date: 12-Mar-2008
https://dl.acm.org/doi/10.1145/1349822.1349849

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten