Skip to main content

‘User as Assessor’ Approach to Embodied Conversational Agents

The Case of Apparent Attention in ECAs

  • Chapter
From Brows to Trust

Part of the book series: Human-Computer Interaction Series ((HCIS,volume 7))

  • 265 Accesses

Abstract

Traditionally, an optimal embodied conversational agent (ECA) has the same capabilities and appearance as an actual person. This chapter proposes a ‘user as assessor’ approach to evaluating ECAs that focuses on how ECAs manifest human capabilities independent of actual capabilities that an ECA may possess. Literatures on humans as producers of behavior and humans as interpreters of behavior are lever-aged to draw implications for how ECAs should behave to seem most realistic to their human assessors. To illustrate the approach, we answer the question, ”what will convince a user that an ECA is paying attention to him or her, whether the ECA truly is paying attention or not?” ‘Apparent attention’ is conceptualized in terms of two basic dimensions — selectivity and breadth — and their indicators and impacts. Using the proposed approach, the chapter provides guidelines for how agents, conversational agents, and ECAs can effectively exhibit attention.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  • Andrée, E., Klesen, M., Gebhard, P., Allen, S. and Rist, T. (1999). Integrating models of personality and emotions into lifelike characters, In Proceedings of the International Workshop on Affect in Interactions: Towards a New Generation of Interfaces, pp. 136–149, Siena, Italy.

    Google Scholar 

  • Argyle, M. and Cook, M. (1976). Gaze and mutual gaze, Cambridge University Press, New York.

    Google Scholar 

  • Bailenson, J.N., Blascovich, J.B., Beall, A.C. and Loomis, J.M. (2001). Equilibrium theory revisited: Mutual gaze and personal space in virtual environments. Presence: Teleoperators and Virtual Environments, 10(6): 583–C598.

    Article  Google Scholar 

  • Bailenson, J. N., Blascovich, J., Beall, A.C. and Loomis, J.M. (in press) Interpersonal distance in immersive virtual environments, Personality and Social Psychology Bulletin.

    Google Scholar 

  • Bavelas, J.B., Black A., Chovil, N., Lemery, C.R., and Mullett, J. (1988). Form and function in motor mimicry: Topographic evidence that the primary function is communicative, Human Communication Research, 14: 275–299.

    Article  Google Scholar 

  • Beaumont, J. G. (1983). Introduction to neuropsychology, Blackwell, Oxford.

    Google Scholar 

  • Beniger, J. R. (1987). Personalization of mass media and the growth of pseudo-community, Communication Research, 14(3): 325–371.

    Article  Google Scholar 

  • Biernet, M. and Vescio, T. K. (1993). Categorization and stereotyping: Effects of group context on memory and social judgment. Journal of Experimental Social Psyhcology, 29: 166–202.

    Article  Google Scholar 

  • Bishop, D. V. M., Chan, J., Hartley, J. and Weir, F. (1998). When a nod is as good as a word: Form-function relationships between questions and their responses. Applied Psycholinguistics, 19(3):415–432.

    Article  Google Scholar 

  • Borkenau, P., and Liebler, A. (1993). Convergence of Stranger Ratings of Personality and Intelligence With Self-Ratings, Partner Ratings, and Measured Intelligence, Journal of Personality and Social Psychology 65:546–653.

    Article  Google Scholar 

  • Branigan, H. P., Pickering, M. J. and Cleland, A. A. (2000) Syntactic coordination in dialogue, Cognition, 75: B13–B25.

    Article  Google Scholar 

  • Brave, S. (2003). Agents that care: Investigating the effects of orientation of emotion exhibited by an embodied computer agent, Unpublished doctoral dissertation, Stanford University.

    Google Scholar 

  • Brave, S. and Nass, C. (2002). Emotion in human-computer interaction. In Jacko, J. and Sears, A. editors, Handbook of human-computer interaction, pp. 251–271, Lawrence Erlbaum Associates, New York.

    Google Scholar 

  • Brennan, S. E. and Clark, H. H. (1996). Conceptual pacts and lexical choice in conversation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22: 1482–1493.

    Google Scholar 

  • Carnegie, D. (1990). How to win friends and influence people, Pocket Books, New York.

    Google Scholar 

  • Cassell, J. (2000). More than just another pretty face: Embodied conversational interface agents. Communications of the ACM, 43(4):70–78.

    Article  Google Scholar 

  • Cassell, J., Nakano, Y.I., Bickmore, T., Sidner, C.L., and Rich, C. (2001). Non-verbal cues for discourse structure. In Proc. of the 41st Annual Meeting of the Association of Computational Linguistics, pp. 106–115. Toulouse, France.

    Google Scholar 

  • Cassell, J. and Stone, M. (1999). Living hand to mouth: Theories of speech and gesture in interactive systems. In Proceedings of AAAI Fall Symposium: Psychological Models of Communication in Collaborative Systems, pp. 34–42, Cape Cod, MA.

    Google Scholar 

  • Chartrand, T.L. and Bargh, J.A. (1999). The chameleon effect: The perception behavior link and social interaction, Journal of Personality and Social Psychology, 76:893–910.

    Article  Google Scholar 

  • Chopra-Khullar S. and Badler N. I. (1999). Where to look? Automating visual attending behaviors of virtual human characters. In Proceedings of ACM Autonomous Agents Conference, pp. 16–23, Seattle, WA.

    Google Scholar 

  • Clark, H. H. (1996). Using Language, Cambridge University Press, New York.

    Book  Google Scholar 

  • Coen, M. (1998). Design principles for intelligent environments. In Proc. of the National Conference on Artificial Intelligence, pp. 547–554, Madison, Wisconsin.

    Google Scholar 

  • Coker, D. A. and Burgoon, J. K. (1987). The nature of conversational involvement and nonverbal encoding patterns. Human Communication Research, 13(4):463–494.

    Article  Google Scholar 

  • Eysenck, M. W. (1997). Principles of cognitive psychology, Psychology Press, New York.

    Google Scholar 

  • Friedman, B. (1997). Human values and the design of computer technology. CSLI Press, Stanford, CA.

    Google Scholar 

  • Giges, B. (1975). Using your head: Notes on nodding. Transactional Analysis Journal, 5(3):264–266.

    Google Scholar 

  • Grayson, D. and Coventry, L. (1998). The effects of visual proxemic information in video mediated communication. SIGCHI Bulletin, 30(3):30–39.

    Article  Google Scholar 

  • Grice, H. P. (1975). Logic and conversation. In Cole, P. and Morgan, J. L., editors, Syntax and semantics, Volume 3: Speech acts. Academic Press, New York.

    Google Scholar 

  • Grosz, B.J and Sidner, C.L. (1986). Attention, intentions, and the structure of discourse. Computational Linguistics, 12(3):175–204.

    Google Scholar 

  • Grosz, B. J., Joshi, A. and Weinstein, S. (1995). Centering: A framework for modelling the local coherence of discourse. Computational Linguistics, 2(21):203–225.

    Google Scholar 

  • Haller, S. M., McRoy, S. W. and Ali, S. S. (1997). Towards a model for mixed initiative in dialogic discourse. In Working Notes of the AAAI Spring Symposium on Computational Models of Mixed Initiative Interaction, pp. 78–80. Stanford University, California.

    Google Scholar 

  • Hoeks, B. and Levelt, W.J.M. (1994). Pupillary dilation as a measure of attention: A quantitative system analysis. Behavior Research Methods, Instruments and Computers, 25: 16–26.

    Article  Google Scholar 

  • Hochschild, A. R. (1985). The managed heart, University of California Press, Berkeley, CA.

    Google Scholar 

  • Höoöok, K., Persson, P., and Sjöolinder, M. (2000). Evaluating Users Experience of a Character Enhanced Information Space, Journal of AI Communications, 13(3):195–212.

    Google Scholar 

  • Isbister, K. and Nass, C. (2000). Consistency of personality in interactive characters: Verbal cues, non-verbal cues, and user characteristics. International Journal of Human-Computer Studies, 53(1):251–267.

    Article  Google Scholar 

  • James, W. (1890). The principles of psychology, New York: Holt.

    Book  Google Scholar 

  • Jeffrey, P. and Mark, G. (1999). Navigating the Virtual Landscape: Coordinating the Shared Use of Space. In Munro, A., Höök, K., and Beynon, D., editors, Social Navigation of Information Space, pp. 112–131, Springer, London.

    Chapter  Google Scholar 

  • Kahneman, D. (1973). Attention and effort, Prentice-Hall, Englewood Cliffs, NJ.

    Google Scholar 

  • Kant, I. (1781 [1929]). The critique of pure reason, Macmillan, London.

    Google Scholar 

  • Kendon, A. (1967). Some functions of gaze direction in social interaction. Acta Psychologica, 32:1–25.

    Google Scholar 

  • Kettebekov, S., Yeasin, M. and Sharma, R. (2002). Prosody based co-analysis for continuous recognition of coverbal gestures, In Proc. of the International Conference on Multimodal Interfaces (ICMI’02), pp. 161–166, Pittsburgh, USA.

    Google Scholar 

  • Krause, M. A. (1997). Comparative perspectives on pointing and joint attention in children and apes. International Journal of Comparative Psychology, 10(3):137–157.

    Google Scholar 

  • Lakoff, R. (1975). Language and woman’s place, Harper and Row, New York.

    Google Scholar 

  • Lashkari, Y., Metral, M. and Maes, P. (1994). Collaborative Interface Agents. In Proc. of the Twelfth National Conference on Artificial Intelligence, pp. 444–449, Menlo Park, CA.

    Google Scholar 

  • Latorella, K.A. (1999). Investigating interruptions: Implications for flightdeck performance. NASA/TM-1999-209707, National Aviation and Space Administration, Washington.

    Google Scholar 

  • Lester, J. C., Voerman, J. L., Towns S. G. and Callaway, C. B. (1997). Cosmo: A life-like animated pedagogical agent with deictic believability. In Working notes of the IJCAI’ 97 workshop on animated interface agents: Making them intelligent, pp. 61–69, Nagoya, Japan.

    Google Scholar 

  • Leung, E. H. and Rheingold, H. L. (1981). Development of pointing as a social gesture. Developmental Psychology, 17(2):215–220.

    Article  Google Scholar 

  • Lieberman, H. (1995). Letizia: An agent that assists in web browsing. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, pp. 457–480, Montreal, Canada.

    Google Scholar 

  • Maglio, P., Barrett, R., Campbell, C. S. and Selker T. (2000). SUITOR: An attentive information system. In Proceedings of the International Conference on Intelligent User Interfaces, pp. 169–176, New Orleans, Louisiana, USA.

    Google Scholar 

  • Maglio, P. P. and Campbell, C. S. (2003). Attentive agents. Communications of the ACM, 46(3):47–51.

    Article  Google Scholar 

  • Mann, W. C. and Thompson, S. A. (1987). Rhetorical structure theory: A theory of text organization, Technical Report RS-87-190, International Computer Science Institute, Marina Del Rey, CA.

    Google Scholar 

  • McNeill, D. (1992). Hand and mind, Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Metral, M. (1993). Design of a generic learning interface agent. Bachelor of Science thesis, Massachusetts Institute of Technology, Cambridge, MA.

    Google Scholar 

  • Minsky, M. (1986). The society of mind, Simon and Schuster, New York.

    Google Scholar 

  • Mirsky, A. F. (1989). The neuropsychology of attention: Elements of a complex behavior. In Perenman, E., editor, Integrating theory and practice in clinical neuropsychology, pp. 75–91, Erlbaum, Hillsdale, NJ.

    Google Scholar 

  • Moon, Y. and Nass, C. (1996). How “real” are computer personalities? Psychological responses to personality types in human-computer interaction. Communication Research, 23(6):651–674.

    Article  Google Scholar 

  • Moon, Y. (1998). Impression management in computer-based interviews: The effects of input modality, output modality, and distance. Public Opinion Quarterly, 62:610–622.

    Article  Google Scholar 

  • Moore, C. and D’Entremont, B. (2001). Developmental changes in pointing as a function of attentional focus. Journal of Cognition and Development, 2(2):109–129.

    Article  Google Scholar 

  • Murray, J. (1991). Anatomy of a new medium: Literary and pedogogic uses of advanced linguistic computer structures. Computers and the Humanities, 25(1):1–14.

    Article  Google Scholar 

  • Nass, C. and Gong, L. (2000). Social aspects of speech interfaces from an evolutionary perspective: Experimental research and design implications. Communications of the ACM, 43(9):36–43.

    Article  Google Scholar 

  • Nass, C. and Lee, K. (2001). Does computer-synthesized speech manifest personality? Experimental tests of recognition, similarity-attraction, and consistency-attraction. Journal of Experimental Psychology: Applied, 7(3):171–181.

    Google Scholar 

  • Nass, C. and Mason, L. (1990). On the study of technology and task: A variable-based approach. In Fulk, J. and C. Steinfeld, C., editors, Organizations and communication technology, pp. 46–67, Sage, Newbury Park.

    Chapter  Google Scholar 

  • Nass, C. and Moon, Y. (2000). Machines and mindlessness: Social responses to computers. Journal of Social Issues, 56(1):81–103.

    Article  Google Scholar 

  • Nass, C., Isbister, K. and Lee, E. (2000). Truth is beauty: Researching embodied conversation agents. In Cassell, J., Sullivan, J., Prevost, S., and Churchill, E., editors, Embodied conversational agents, pp. 374–402, MIT Press, Cambridge, MA.

    Google Scholar 

  • Nass, C., Moon, Y. and Green, N. (1997). Are computers gender-neutral? Gender stereotypic responses to computers. Journal of Applied Social Psychology, 27(10):864–876.

    Article  Google Scholar 

  • Norman., D. A. (1997). How might people interact with agents. pp. 49–55. In Bradshaw, J. M., editor, Software agents, AAAI Press/The MIT Press, Cambridge, MA.

    Google Scholar 

  • Oh, A., Fox, H., Kleek, M. V., Adler, A., Gajos, K., Morency, L. and Darrell, T. (2002). Evaluating look-to-talk: A gaze-aware interface in a collaborative environment. In Proc. of the Conference on Human Factors in Computing Systems (CHI), pp. 650–651. Minneapolis, MN.

    Google Scholar 

  • Persson, P., Laaksolahti, J., and Löonnqvist, P. (2000). Anthropomorphism: A multi-layered phenomenon. AAAI Fall Symposium 2000, Technical Report FS-00-04, pp. 131–135, North Falmouth, Massachusetts.

    Google Scholar 

  • Poesio, M. (1992). Conversational events and discourse state change: A preliminary report. In Proceedings of the 3rd International Conference on Principles of Knowledge Representation and Reasoning (KR’92), pp. 369–380, Cambridge, MA.

    Google Scholar 

  • Reed, C. A. (2002). Saliency and the attentional state in natural language generation. In Proceedings of the 15th European Conference on Artificial Intelligence, pp. 440–444, Lyon, France.

    Google Scholar 

  • Reeves, B. and Nass, C. (1996). The media equation: How people treat computers, television, and new media like real people and places, Cambridge University Press/CSLI, New York.

    Google Scholar 

  • Rich, C. and Sidner, C. (1998). COLLAGEN: A collaboration manager for software interface agents. User Modeling and User Adapted Interaction, 8:315–350.

    Article  Google Scholar 

  • Rickel, J., Lesh, N. B., Rich, C., Sidner, C. L. and Gertner, A. (2002). Collaborative discourse theory as a foundation for tutorial dialogue. In Proc. of the Conference on Intelligent Tutoring Systems, pp. 542–551, Biarritz, France and San Sebastian, Spain.

    Google Scholar 

  • Rickel, J. and Johnson, W. L. (1999). Animated agents for procedural training in virtual reality: Perception, cognition, and motor control. Applied Artificial Intelligence, 13(4–5):343–382.

    Article  Google Scholar 

  • Rizzo P., Veloso M. V., Miceli M. and Cesta A. (1997). Personality-driven social behaviors in believable agents. In Proc. of the AAAI 1997 Fall Symposium on “Socially Intelligent Agents”, AAAI Press Technical Report FS-97-02, pp. 109–114.

    Google Scholar 

  • Rosenkrantz, P. S., Vogel, S. R., Bee, H., Broverman, I. K. and Broverman, D. M. (1968). Sex role stereotypes and self concepts in college students. Journal of Consulting and Clinical Psychology, 32:287–295.

    Article  Google Scholar 

  • Schmidt, C. L. (1999). Adult understanding of spontaneous attention-directing events: What does gesture contribute? Ecological Psychology, 11(2):139–174.

    Article  Google Scholar 

  • Schneiderman, B. (1997). Designing the user interface: Strategies for e ective human-computer interaction, Addison Wesley, New York.

    Google Scholar 

  • Shankar, T. R., van Kleek, M., Vicente, A. and Smith, B. K. (2000). Fugue: A computer mediated conversational system that supports turn negotiation. In Proc. of the Thirty-Third Hawaii International Conference on System Sciences Persistent Conversation mini-track, pp. 3035–3036, Maui, Hawaii.

    Google Scholar 

  • Sherman, E. (1973). Listening comprehension as a function of proxemic distance and eye-contact. Graduate Research in Education and Related Disciplines, 7(1):5–34.

    Google Scholar 

  • Sheth, B. and Maes, P. (1993). Evolving agents for personalized information filtering. In Proc. of the Ninth IEEE Conference on Artificial Intelligence and its Applications, pp. 1–7. Orlando, FL.

    Google Scholar 

  • Spence, C., Kettenmann, B., Kobal, G., McGlone, F.P. (2001). Shared attentional resources for processing visual and chemosensory information. Quarterly Journal of Experimental Psychology, 54A(3):775–783.

    Article  Google Scholar 

  • Tajfel, H. (Ed.) (1982). Social identity and intergroup behavior, Cambridge: Cambridge University Press.

    Google Scholar 

  • Tannen, D. (1990). You just don’t understand: Men and women in conversation, Ballantine, New York.

    Google Scholar 

  • Traum, D. R. and Rickel, J. (2002). Embodied agents for multi-party dialogue in immersive virtual worlds. In Proc. of the First International Joint Conferenceon Autonomous Agents and Multi-Agent System, pp. 766–773, Bologna, Italy.

    Google Scholar 

  • Vertegaal, R. and Ding, Y. (2003). Explaining effects of eye gaze on mediated group conversations: Amount or synchronization? In Proceedings of Computer Supported Cooperative Work, pp. 41–48, New Orleans, Louisiana.

    Google Scholar 

  • Vertegaal, R., Slagter, R., Van der Veer, G. C. and Nijholt, A. (2001). Eye gaze patterns in conversations: There is more to conversational agents than meets the eyes. In Proceedings of ACM CHI 2001 Conference on Human Factors in Computing Systems, pp. 301–308, Seattle, Washington.

    Google Scholar 

  • Vertegaal, R., Veer, G. and Vons, H. (2002). Effects of gaze on multiparty mediated communication.In Proceedings of Graphics Interface, pp. 95–102, Montreal, Canada.

    Google Scholar 

  • Vilhjalmsson, H. and Cassell, J. (1998). BodyChat: Autonomous communicative behaviors in avatars, In Proceedings of ACM International Conference on Autonomous Agents, pp. 269–276, Minneapolis.

    Google Scholar 

  • Walker, M. A. (1996). Limited attention and discourse structure. Computational Linguistics, 22(2):255–264.

    Google Scholar 

  • Weizenbaum, J. (1966). Eliza — a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9:23–28.

    Article  Google Scholar 

  • Wickens, C. D. (1984). Processing resources in attention. In Parasuraman, R. and Davies, D. R., editors, Varieties of attention, Academic Press, New York.

    Google Scholar 

Download references

Authors

Editor information

Zsófia Ruttkay Catherine Pelachaud

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Kluwer Academic Publishers

About this chapter

Cite this chapter

Nass, C., Robles, E., Wang, Q. (2004). ‘User as Assessor’ Approach to Embodied Conversational Agents. In: Ruttkay, Z., Pelachaud, C. (eds) From Brows to Trust. Human-Computer Interaction Series, vol 7. Springer, Dordrecht. https://doi.org/10.1007/1-4020-2730-3_6

Download citation

  • DOI: https://doi.org/10.1007/1-4020-2730-3_6

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-1-4020-2729-1

  • Online ISBN: 978-1-4020-2730-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics