Abstract
We describe an anthropomorphic agent that is engaged in an imitation game with the human user. In imitating natural gestures demonstrated by the user, the agent brings together gesture recognition and synthesis on two levels of representation. On the mimicking level, the essential form features of the meaning-bearing gesture phase (stroke) are extracted and reproduced by the agent. Meaning-based imitation requires extracting the semantic content of such gestures and re-expressing it with possibly alternative gestural forms. Based on a compositional semantics for shape-related iconic gestures, we present first steps towards this higher-level gesture imitation in a restricted domain.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjalmsoon, H., Yan, H.: Embodiment on conversational interfaces: Rea. In: CHI 1999 Conference, pp. 520–527. ACM Press, New York (1999)
Cassell, J., Stone, M., Yan, H.: Coordination and context-dependence in the generation of embodied conversation. In: Proceedings of the International Natural Language Generation Conference, Mitzpe Ramon, Israel, pp. 171–178 (June 2000)
Gibet, S., Lebourque, T., Marteau, P.: High-level specification and animation of communicative gestures. Journal of Visual Languages and Computing 12(6), 657–687 (2001)
Johnston, M.: Multimodal unification-based grammars. Technical Report WS-98- 09. AAAI Press, Menlo Park (1998)
Kita, S., van Gijn, I., van der Hulst, H.: Movement phases in signs and cospeech gestures, and their transcription by human coders. In: Wachsmuth, I., Fröhlich, M. (eds.) GW 1997. LNCS (LNAI), vol. 1371, pp. 23–36. Springer, Heidelberg (1998)
Kopp, S., Wachsmuth, I.: A knowledge-based approach for lifelike gesture animation. In: Horn, W. (ed.) ECAI 2000 Proceedings of the 14th European Conference on Artificial Intelligence, pp. 661–667. IOS Press, Amsterdam (2000)
Kopp, S., Wachsmuth, I.: Model-based animation of coverbal gesture. In: Proc. of Computer Animation 2002, Los Alamitos, CA, pp. 252–257. IEEE Computer Society Press, Los Alamitos (2002)
Lang, E.: The semantics of dimensional designation of spatial objects. In: Bierwisch, M., Lang, E. (eds.) Dimensional Adjectives: Grammatical Structure and Conceptual Interpretation, pp. 263–417. Springer, Heidelberg (1989)
Latoschik, M.E.: A general framework for multimodal interaction in virtual reality systems: PrOSA. In: VR 2001 workshop proceedings: The Future of VR and AR Interfaces: Multi-modal, Humanoid, Adaptive and Intelligent (2001) (in press)
Matarić, M., Zordan, V., Williamson, M.: Making complex articulated agents dance. Autonomous Agents and Multi-Agent Systems 2(1), 23–44 (1999)
Prillwitz, S., Leven, R., Zienert, H., Hamke, T., Henning, J.: HamNoSys Version 2.0: Hamburg Notation System for Sign Languages: An Introductory Guide. International Studies on Sign Language and Communication of the Deaf, vol. 5. Signum Press, Hamburg (1989)
Rickel, J., Johnson, W.: Animated agents for procedural training in virtual reality: Perception, cognition, and motor control. Applied Artificial Intelligence 13, 343–382 (1999)
Sowa, T., Wachsmuth, I.: Interpretation of shape-related iconic gestures in virtual environments. In: Wachsmuth, I., Sowa, T. (eds.) GW 2001. LNCS (LNAI), vol. 2298, pp. 21–33. Springer, Heidelberg (2002)
Sowa, T., Wachsmuth, I.: Coverbal Iconic Gestures for Object Descriptions in Virtual Environments: An Empirical Study. In: Rector, M., Poggi, I., Trigo, N. (eds.) Proceedings of the Conference Gestures, Meaning and Use, Porto, Portugal, pp. 365–376 (2003); Edições Universidade Fernando Pessoa
Sparrell, C.J., Koons, D.B.: Interpretation of coverbal depictive gestures. In: AAAI Spring Symposium Series: Intelligent Multi-Media Multi-Modal Systems, pp. 8–12. Stanford University, Stanford (1994)
Wu, Y., Huang, T.S.: Vision-based gesture recognition: A review. In: Braffort, A., Gibet, S., Teil, D., Gherbi, R., Richardson, J. (eds.) GW 1999. LNCS (LNAI), vol. 1739, pp. 103–115. Springer, Heidelberg (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kopp, S., Sowa, T., Wachsmuth, I. (2004). Imitation Games with an Artificial Agent: From Mimicking to Understanding Shape-Related Iconic Gestures. In: Camurri, A., Volpe, G. (eds) Gesture-Based Communication in Human-Computer Interaction. GW 2003. Lecture Notes in Computer Science(), vol 2915. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24598-8_40
Download citation
DOI: https://doi.org/10.1007/978-3-540-24598-8_40
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21072-6
Online ISBN: 978-3-540-24598-8
eBook Packages: Springer Book Archive