Abstract
Individuality of Embodied Conversational Agents (ECAs) may depend on both the look of the agent and the way it combines different modalities such as speech and gesture. In this chapter, we describe a study in which male and female users had to listen to three short technical presentations made by ECAs. Three multimodal strategies of ECAs for using arm gestures with speech were compared: redundancy, complementarity, and speech-specialization. These strategies were randomly attributed to different-looking 2D ECAs, in order to test independently the effects of multimodal strategy and ECA’s appearance. The variables we examined were subjective impressions and recall performance. Multimodal strategies proved to influence subjective ratings of quality of explanation, in particular for male users. On the other hand, appearance affected likeability, but also recall performance. These results stress the importance of both multimodal strategy and appearance to ensure pleasantness and effectiveness of presentation ECAs.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
AAMAS (2002). Marriott, A., Pelachaud, C., Rist, T., Ruttkay, Zs., and Vilhjálmsson, H., editors. Proceedings of Workshop on Embodied Conversational Agents — Let’s Specify and Evaluate them!. AAMAS02, Bologna, Italy.
AAMAS (2003). Pelachaud, C., Marriott, A., and Ruttkay, Zs., editors. Proceedings of Workshop on Embodied Conversational Characters as Individuals. AAMAS03, Melbourne, Australia.
Abrilian, S., Buisine, S., Rendu, C., and Martin, J.-C. (2002). Specifying Cooperation between Modalities in Lifelike Animated Agents. In Proc. PRICAI02 Workshop on Lifelike Animated Agents: Tools, Functions, and Applications, pp. 3–8, Tokyo, Japan.
André, E., Rist, T., Van Mulken, S., Klesen, M., and Baldes, S. (2000). The automated design of believable dialogues for animated presentation teams. In Cassell, J., Prevost, S., and Churchill, E., editors, Embodied Conversational Agents. pp. 220–255, MIT Press, Cambridge.
Ball, G. and Breese, J. (2000). Emotion and personality in a conversational character. In Cassell, J., Prevost, S., and Churchill, E., editors, Embodied Conversational Agents. pp. 189–219, MIT Press, Cambridge.
Cassell, J., Bickmore, T., Vilhjálmsson, H., and Yan, H. (2001). More than just a pretty face: Conversational protocols and the affordances of embodiment. Knowledge-Based Systems, 14: 55–64.
Cassell, J. and Stone, M. (1999). Living hand to mouth: Psychological theories about speech and gesture in interactive dialogue systems. In Proc. of AAAI99, pp. 34–42, North Falmouth, MA.
Craig, S. D., Gholson, B., and Driscoll, D. (2002). Animated pedagogical agents in multimedia educational environments: Effects of agent properties, picture features, and redundancy. Journal of Educational Psychology, 94: 428–434.
Feldman, R. S., Philippot, P., and Custrini, R. J. (1991). Social competence and nonverbal behavior. In Feldman, R.S. and Rimé, B., editors, Fundamentals of Nonverbal Behavior. pp. 329–350, Cambridge University Press, Cambridge, England.
Giese, M.A. and Poggio, T. (2003). Neural mechanisms for recognition of biological movements. Nature Reviews, 4: 179–192.
Goldin-Meadow, S. (1999a). The role of gesture in communication and thinking. Trends in Cognitive Sciences, 3(11): 419–429.
Goldin-Meadow, S., Kim, S., and Singer, M. (1999b). What the teacher’s hands tell the student’s mind about math. Journal of Educational Psychology, 91: 720–730.
Kendon, A. (1980). Gesticulation and speech: two aspects of the process of utterance. In Key, M.R., editor, The relationship of Verbal and Nonverbal Communication. pp. 207–228, Mouton Publishers, The Hague, The Netherlands.
Kimura, D. (1999). Sex and Cognition. MIT Press, Cambridge.
Kohar, H. and Ginn, I. (1997). Mediators: Guides through online TV services. In Electronic Proc. CHI97. URL: http://www.acm.org/sigchi/chi97/proceedings/demo/hk.htm/
Martin, J. C., Grimard, S., and Alexandri, K. (2001). On the annotation of the multimodal behavior and computation of cooperation between modalities. In Proc. AAMAS01 Workshop on Representing, Annotating, and Evaluating Non-Verbal and Verbal Communicative Acts to Achieve Contextual Embodied Agents, pp. 1–7, Montreal, Canada.
McBreen, H., Anderson, J., and Jack, M. (2001). Evaluating 3D embodied conversational agents in contrasting VRML retail applications. In Proc. Int. Conf. on Autonomous Agents Workshop on Multimodal Communication and Context in Embodied Agents, pp. 83–87, Montreal, Canada.
McNeill, D. (1987). Psycholinguistics: A new approach. Harper and Row, New York.
McNeill, D. (1992). Hand and Mind, University of Chicago Press, Chicago, IL.
Moreno, K. N., Klettke, B., Nibbaragandla, K., and Graesser, A. C. (2002). Perceived characteristics and pedagogical effcacy of animated conversational agents. In Proc. of ITS02, pp. 963–971, Biarritz, France and San Sebastian, Spain.
Moreno, R. and Mayer, R. E. (2002). Verbal redundancy in multimedia learning: when reading helps listening. Journal of Educational Psychology, 94: 156–163.
van Mulken, S. André, E., and Müller, J. (1998). The Persona effect: How substantial is it? In Proc. of HCI98, pp. 53–66, Berlin, Germany.
Myers, J.L. (1979). Fundamentals of Experimental Design. Third Edition, Allyn and Bacon Inc, Boston.
Nijholt, A. (2001). Towards multi-modal emotion display in embodied agents. In Proc. of Artificial Neural Networks and Expert Systems 2001, pp. 229–231, Dunedin, New Zealand.
Pelachaud, C., Carofiglio, V., De Carolis, B., De Rosis, F., and Poggi, I. (2002). Embodied contextual agent in information delivering application. In Proc. of AAMAS02, pp. 758–765, Bologna, Italy.
Rickel, J. and Johnson, W. L. (1999). Animated agents for procedural training in virtual reality: Perception, cognition, and motor control. Applied Artificial Intelligence, 13: 343–382.
Rimé, B. and Schiaratura, L. (1991). Gesture and speech. In: Feldman, R.S. and Rimé, B., editors. Fundamentals of Nonverbal Behavior. pp. 239–284, Cambridge University Press, Cambridge, England.
Editor information
Rights and permissions
Copyright information
© 2004 Kluwer Academic Publishers
About this chapter
Cite this chapter
Buisine, S., Abrilian, S., Martin, JC. (2004). Evaluation of Multimodal Behaviour of Embodied Agents. In: Ruttkay, Z., Pelachaud, C. (eds) From Brows to Trust. Human-Computer Interaction Series, vol 7. Springer, Dordrecht. https://doi.org/10.1007/1-4020-2730-3_8
Download citation
DOI: https://doi.org/10.1007/1-4020-2730-3_8
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-2729-1
Online ISBN: 978-1-4020-2730-7
eBook Packages: Computer ScienceComputer Science (R0)