Abstract
This paper describes an international effort to unify a multimodal behavior generation framework for Embodied Conversational Agents (ECAs). We propose a three stage model we call SAIBA where the stages represent intent planning, behavior planning and behavior realization. A Function Markup Language (FML), describing intent without referring to physical behavior, mediates between the first two stages and a Behavior Markup Language (BML) describing desired physical realization, mediates between the last two stages. In this paper we will focus on BML. The hope is that this abstraction and modularization will help ECA researchers pool their resources to build more sophisticated virtual humans.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Cassell, J., Pelachaud, C., Badler, N., Steedman, M., Achorn, B., Becket, T., Douville, B., Prevost, S., Stone, M.: Animated Conversation: Rule-Based Generation of Facial Expression, Gesture and Spoken Intonation for Multiple Conversational Agents. In: Siggraph 1994 Conference Proceedings, ACM SIGGRAPH, pp. 413–420. Addison Wesley, Reading (1994)
Cassell, J., Vilhjálmsson, H., Bickmore, T.: BEAT: the Behavior Expression Animation Toolkit. In: Proc. ACM SIGGRAPH 2001, Los Angeles, August 12-17, pp. 477–486 (2001)
Cassell, J., Vilhjálmsson, H., Chang, K., Bickmore, T., Campbell, L., Yan, H.: Requirements for an Architecture for Embodied Conversational Characters. In: Computer Animation and Simulation 1999. Eurographics Series. Springer, Austria (1999)
DeCarolis, B., Pelachaud, C., Poggi, I., Steedman, M.: APML, a mark-up language for believable behavior generation. In: Prendinger, H., Ishizuka, M. (eds.) Life-like Characters. Tools, Affective Functions and Applications, pp. 65–85. Springer, Heidelberg (2004)
Hartmann, B., Mancini, M., Pelachaud, C.: Formational parameters and adaptive prototype instantiation for MPEG-4 compliant gesture synthesis. In: Computer Animation 2002, Geneva, Switzerland. IEEE Computer Society Press, Los Alamitos (2002)
Kopp, S., Jung, B., Lessmann, N., Wachsmuth, I.: Max–A Multimodal Assistant in Virtual Reality Construction. KI 4/03, 11–17 (2003)
Kopp, S., Wachsmuth, I.: Synthesizing Multimodal Utterances for Conversational Agents. Computer Animation and Virtual Worlds 15(1), 39–52 (2004)
Krenn, B.: Representational Lego for ECAs. In: Background paper for a presentation held at the FP6 NoE HUMAINE Workshop on Emotion and Interaction, Paris (March 10-11, 2005)
Krenn, B., Pirker, H.: Defining the Gesticon: Language and Gesture Coordination for Interacting Embodied Agents. In: Krenn, B., Pirker, H. (eds.) Proc. of the AISB 2004 Symposium on Language, Speech and Gesture for Expressive Characters, University of Leeds, UK, pp. 107–115 (2004)
Martell, C.: FORM: An Extensible, Kinematically-based Gesture Annotation Scheme. In: Proceedings of the 2002 International Conference on Language Resources and Evaluation, Las Palmas, Canary Island (2002)
Piwek, P., Krenn, B., Schröder, M., Grice, M., Baumann, S., Pirker, H.: RRL: A Rich Representation Language for the Description of Agent Behaviour in NECA. In: Proceedings of the Workshop Embodied conversational agents - let’s specify and evaluate them!, held in conjunction with AAMAS 2002, Bologna, Italy (July 16, 2002)
Prendinger, H., Descamps, S., Ishizuka, M.: MPML: A markup language for controlling the behavior of life-like characters. Journal of Visual Languages and Computing 15(2), 183–203 (2004)
Stokoe, W.C., Casterline, D.C., Croneberg, C.G.: A dictionary of American sign language on linguistic principles. Linstok Press (1976)
Searle, J.R.: Speech acts: An essay in the philosophy of language. Cambridge Univ. Press, London (1969)
Thórisson, K.R.: Dialogue Control in Social Interface Agents. In: InterCHI Adjunct Proceedings 1993, Amsterdam, pp. 139–140 (April 1993)
Thórisson, K.R.: Computational Characteristics of Multimodal Dialogue. In: AAAI Fall Symposium on Embodied Language and Action, Massachusetts Institute of Technology, Cambridge, Massachusetts, November 10-12, pp. 102–108 (1995)
Thórisson, K.R.: A Mind Model for Multimodal Communicative Creatures and Humanoids. International Journal of Applied Artificial Intelligence 13(4-5), 449–486 (1999)
Thórisson, K.R., Vilhjalmsson, H., Kopp, S., Pelachaud, C.: Report on Representations for Multimodal Generation Workshop. AI Magazine 27(1), 108 (2006)
Vilhjalmsson, H.: Animating Conversation in Online Games. In: Rauterberg, M. (ed.) ICEC 2004. LNCS, vol. 3166, pp. 139–150. Springer, Heidelberg (2004)
Vilhjalmsson, H.: Augmenting Online Conversation through Automated Discourse Tagging. In: 6th annual minitrack on Persistent Conversation at the 38th Hawaii International Conference on System Sciences, Hilton Waikoloa Village, Big Island, Hawaii, January 3-6. IEEE, Los Alamitos (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kopp, S. et al. (2006). Towards a Common Framework for Multimodal Generation: The Behavior Markup Language. In: Gratch, J., Young, M., Aylett, R., Ballin, D., Olivier, P. (eds) Intelligent Virtual Agents. IVA 2006. Lecture Notes in Computer Science(), vol 4133. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11821830_17
Download citation
DOI: https://doi.org/10.1007/11821830_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-37593-7
Online ISBN: 978-3-540-37594-4
eBook Packages: Computer ScienceComputer Science (R0)