Abstract
We present a collaborative approach towards a detailed understanding of the usage of pointing gestures accompanying referring expressions. This effort is undertaken in the context of human-machine interaction integrating empirical studies, theory of grammar and logics, and simulation techniques. In particular, we attempt to measure the precision of the focussed area of a pointing gesture, the so-called pointing cone. The pointing cone serves as a central concept in a formal account of multi-modal integration at the linguistic speech-gesture interface as well as in a computational model of processing multi-modal deictic expressions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
André, E., Rist, T., Müller, J.: Employing AI Methods to Control the Behavior of Animated Interface Agents. Applied Artificial Intelligence 13, 415–448 (1999)
Claassen, W.: Generating Referring Expressions in a Multimodal Environment. In: Dale, R., Rösner, D., Stock, O., Hovy, E., et al. (eds.) IWNLG 1992. LNCS, vol. 587, pp. 247–262. Springer, Heidelberg (1992)
Dale, R.: Cooking up Referring Expressions. In: Proceedings of the 27th Annual Meeting of the ACL, Vancouver, pp. 68–75 (1989)
Dale, R., Reiter, E.: Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions. Cognitive Science 18, 233–263 (1995)
Kendon, A.: Gesture: Visible Action as Utterance. Cambridge University Press, Cambridge (2004)
Kita, S. (ed.): Pointing. Where Language, Culture, and Cognition Meet. Lawrence Erlbaum Associates, Mahwah NJ (2002)
Krahmer, E., van der Sluis, I.: A New Model for the Generation of Multimodal Referring Expressions. In: Proceedings European Workshop on Natural Language Generation (ENLG 2003), Budapest, pp. 47–54 (2003)
Kopp, S., Wachsmuth, I.: Synthesizing Multimodal Utterances for Conversational Agents. Comp. Anim. Virtual Worlds 15, 39–52 (2004)
Kranstedt, A., Kühnlein, P., Wachsmuth, I.: Deixis in Multimodal Human Computer Interaction: An Interdisciplinary Approach. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 112–123. Springer, Heidelberg (2004)
Kranstedt, A., Wachsmuth, I.: Incremental Generation of Multimodal Deixis Referring to Objects. In: Proceedings European Workshop on Natural Language Generation (ENLG 2005), Aberdeen, UK, pp. 75–82 (2005)
Latoschik, M.E.: A General Framework for Multimodal Interaction in Virtual Reality Systems: PrOSA. In: Proceedings of the Workshop The Future of VR and AR Interfaces - Multimodal, Humanoid, Adaptive and Intelligent. IEEE Virtual Reality 2001, Yokohama, pp. 21–25 (2001)
Lester, J., Voerman, J., Towns, S., Callaway, C.: Deictic Believability: Coordinating Gesture, Locomotion, and Speech in Lifelike Pedagogical Agents. Applied Artificial Intelligence 13(4-5), 383–414 (1999)
Lücking, A., Rieser, H., Stegmann, J.: Statistical Support for the Study of Structures in Multimodal Dialogue: Inter-rater Agreement and Synchronisation. In: Proceedings of the 8th Workshop on the Semantics and Pragmatics of Dialogue (Catalog 2004), pp. 56–64. Univ. Pompeu Fabra, Barcelona (2004)
McNeill, D.: Hand and Mind: What Gestures Reveal about Thought. University of Chicago Press, Chicago (1992)
Pfeiffer, T., Latoschik, M.E.: Resolving Object References in Multimodal Dialogues for Immersive Virtual Environments. In: Proceedings of the IEEE Virtual Reality 2004, Chicago, pp. 35–42 (2004)
Reithinger, N.: The Performance of an Incremental Generation Component for Multimodal Dialog Contributions. In: Dale, R., Rösner, D., Stock, O., Hovy, E., et al. (eds.) IWNLG 1992. LNCS, vol. 587, pp. 263–276. Springer, Heidelberg (1992)
Rieser, H.: Pointing in Dialogue. In: Catalog 2004. Op. cit., pp. 93–101 (2004)
Tramberend, H.: Avocado: A Distributed Virtual Reality Framework. In: Proceedings of IEEE Virtual Reality 1999, pp. 14–21 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kranstedt, A., Lücking, A., Pfeiffer, T., Rieser, H., Wachsmuth, I. (2006). Deixis: How to Determine Demonstrated Objects Using a Pointing Cone. In: Gibet, S., Courty, N., Kamp, JF. (eds) Gesture in Human-Computer Interaction and Simulation. GW 2005. Lecture Notes in Computer Science(), vol 3881. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11678816_34
Download citation
DOI: https://doi.org/10.1007/11678816_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-32624-3
Online ISBN: 978-3-540-32625-0
eBook Packages: Computer ScienceComputer Science (R0)