Abstract
A strong debate has ensued in the computing community about whether Embodied Conversational Agents (ECAs) are beneficial and whether we should pursue this direction in interface design. Proponents cite the naturalness and power of ECAs as strengths, and detractors feel that ECAs disempower, mislead, and confuse users. As this debate rages on, relatively little systematic empirical evaluation on ECAs is actually being performed, and the results from this research have been contradictory or equivocal. We propose a framework for evaluating ECAs that can systematize the research. The framework emphasizes features of the agent, the user, and the task the user is performing. Our goal is to be able to make informed, scientific judgments about the utility of ECAs in user interfaces. If intelligent agents can be built, are there tasks or applications for which an ECA is appropriate? Are there characteristics (in appearance, in personality, etc.) the ECA should have? What types of users will be more productive and happy by interacting with an ECA? Our initial experiment within this framework manipulated the ECA’s appearance (realistic human versus iconic object) and the objectivity of the user’s task (editing a document versus deciding what to pack on a trip). We found that the perception of the ECA was strongly influenced by the task while features of the ECA that we manipulated had littlee ect.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barker, T. (2003). The Illusion of Life Revisited. In Proceedings of AA-MAS 2003 Workshop on Embodied Conversational Characters as In-dividuals, Melbourne, Australia.
Bradshaw, J.M. (1997). An introduction to software agents. In Bradshaw J.M., editor, Software Agents, pp. 3–46, AAAI Press/MIT Press.
Cassell, J. (2000). Embodied conversational interface agents. Communications of the ACM, 43(4): 70–78.
Cassell, J., Sullivan, J., Prevost, S., and Churchill, E., editors (2000). Embodied Conversational Agents, MIT Press, Cambridge.
Cassell, J. and Bickmore, T. (2001). A Relational Agent: A Model and Implementation of Building User Trust. In Proceedings of the ACM CHI 2001 Conference, pp. 396–403, Seattle, Washington.
Collier, G. (1985). Emotional Expression, Lawrence Erlbaum Assoc., Hillsdale, NJ.
Dahlback, N., Jonsson, A., and Ahrenberg, L. (1993). Wizard of Oz studies — why and how. In Proceedings of the 1993 International Workshop on Intelligent User Interfaces, pp. 193–200, Orlando, FL.
Dehn, D.M. and Van Mulken, S. (2000). The impact of animated interface agents: A review of empirical research. International Journal of Human-Computer Studies, 52(1): 1–22.
Erickson, T. (1997). Designing agents as if people mattered. In Bradshaw J.M., editor, Software Agents, pp. 79–96, AAAI Press/MIT Press.
Griffin, P., Hodgson, P., and Prevost, S. (2003). Character User Interfaces for Commercial Applications. In Proceedings of AAMAS 2003 Workshop on Embodied Conversational Characters as Individuals, Melbourne, Australia.
Haddah, H. and Klobas, J. (2003). The Relationship between Visual Abstraction and the Effectiveness of a Pedagogical Character-Agent. In Proceedings of AAMAS 2002 Workshop on Embodied Conversational Agents — Let’s Specify and Evaluate Them!, Bologna, Italy.
Isbister, K. and Doyle, P. (2002). Design and Evaluation of Embodied Conversational Agents: A Proposed Taxonomy. In Proceedings of AAMAS 2002 Workshop on Embodied Conversational Agents — Let’s Specify and Evaluate Them!, Bologna, Italy.
King, W.J. and Ohya, J. (1996). The representation of agents: Anthropomorphism, agency and intelligence. In Proceedings of CHI 1996 Conference Companion, pp. 289–290, Vancouver, B.C.
Koda, T. (1996). Agents with faces: A study on the e ect of personification of software agents. MIT Media Lab, MS thesis.
Lai, J., Wood, D., and Considine, M. (2000). The e ect of task conditions on the comprehensibility of synthetic speech. In Proceedings of the ACM CHI 2000 Conference, pp. 321–328, The Hague, Netherlands.
Lanier, J. (1995). Agents of alienation. Interactions 2(3): 66–72.
Laurel, B. (1990). Interface agents: Metaphors with character. In Laurel B., editor, The art of human-computer interface design, pp. 355–365, Addison-Wesley, New York.
Lyman, P. and Varian, H. (2002). How Much Information?, available at http://www.sims.berkeley.edu/how-much-info/.
Martin, J.C., Grimard, S., and Alexandri, K. (2001). On the annotation of multimodal behavior and the computation of cooperation between modalities. In Proceedings of AAMAS 2001 Workshop on Representing, Annotating, and Evaluating Nonverbal and Verbal Communicative Acts to Achieve Contextual Embodied Agents, pp. 1–7, Montreal, Canada, 2001.
Mayer, R. E., Moreno, R., Boire, M., and Vagge, S. (1999). Maximizing constructivist learning from multimedia communications by minimizing cognitive load. Journal of Educational Psychology, 91(4): 638–643.
McBreen, H.M., Shade, P. Jack, M.A., and Wyard, P.J. (2000). Experimental Assessment of the Effectiveness of Synthetic Personae for Multi-Modal E-Retail Applications. Proceedings of Fourth International Conferenceon Autonomous Agents, pp. 39–45, Barcelona, Spain.
McCrae, R. and Costa P. (1987). Validation of the five-factor model of personality across instruments and observers. Journal of Personality and Social Psychology, 52(1): 81–90.
Moon, Y. and Nass, C. (1996). How ‘real’ are computer personalities? Psychological responses to personality types in human-computer interaction. Communication Research, 23(6): 651–674.
Nass, C., Steuer, J., and Tauber, E. (1994). Computers are social actors. In Proceedings of the ACM CHI 1994 Conference, pp. 72–78, Boston, MA.
Nass, C., Lombard, M., Henriksen, L., and Steuer J. (1995). Anthropocentrism and computers. Behaviour and Information Technology, 14(4): 229–238.
Nass, C. and Lee, K.M. (2000). Does computer-generated speech manifest personality? An experimental test of similarity-attraction. In Proceedings of the ACM CHI 2000 Conference, pp. 329–336, The Hague, Netherlands.
Shneiderman, B. and Maes, P. (1997). Direct manipulation vs. interface agents. Interactions, 4(6): 42–61.
Shneiderman, B. (1997). Direct manipulation versus agents: Paths to predictable, controllable, and comprehensible interfaces. In Bradshaw J.M., editor, Software Agents, pp. 97–106, AAAI Press/ The MIT Press, Menlo Park, CA.
Rickenberg, R. and Reeves, B. (2000). The effects of animated characters on anxiety, task performance, and evaluations of user interfaces. In Proceedings of the ACM CHI 2000 Conference, pp. 329–336, The Hague, Netherlands.
Ruttkay, Z., Dormann, C., and Noot, H. (2002). Evaluating ECAs — What and How?. In Proceedings of AAMAS 2002 Workshop on Embodied Conversational Agents — Let’s Specify and Evaluate Them!, Bologna, Italy.
Sanders, G. and Scholtz, J. (2000). Measurement and Evaluation of Conversational Agents. In Cassell J., Sullivan J., Prevost S., Churchill E., editors, Embodied Conversational Agents, pp. 346–373, MIT Press, Cambridge, MA.
Sculley, J. (1989). The Relationship Between Business and Higher Education: a Perspective on the 21st Century. Communications of the ACM, 32(9): 1056–1061.
Sweller, J., van Merrienboer, J. J. G., and Paas, F. G. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3): 251–296.
Takeuchi A. and Naito, T. (1995). Situated facial displays: Towards social interaction. In Proceedings of the ACM CHI 1995 Conference, pp. 450–455, Denver, CO.
Van Mulken, S., Andre, E., and Muller, J. (1998). The Persona E ect: How substantial is it?. In Proceedings of the ACM CHI 1998 Conference, pp. 53–66, Los Angeles, CA.
Walker, J. H., Sproull, L., and Subramani, R. (1994). Using a human face in an interface. In Proceedings of the ACM CHI 1994 Conference, pp. 85–91, Boston, MA.
Xiao, J., Catrambone, R., and Stasko, J., (2003). Be Quiet? Evaluating Proactive and Reactive User Interface Assistants. In Proceedings of INTERACT’ 03, pp. 383–390, Zurich, Switzerland.
Editor information
Rights and permissions
Copyright information
© 2004 Kluwer Academic Publishers
About this chapter
Cite this chapter
Catrambone, R., Stasko, J., Xiao, J. (2004). ECA as User Interface Paradigm. In: Ruttkay, Z., Pelachaud, C. (eds) From Brows to Trust. Human-Computer Interaction Series, vol 7. Springer, Dordrecht. https://doi.org/10.1007/1-4020-2730-3_9
Download citation
DOI: https://doi.org/10.1007/1-4020-2730-3_9
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-2729-1
Online ISBN: 978-1-4020-2730-7
eBook Packages: Computer ScienceComputer Science (R0)