skip to main content
10.1145/1322192.1322234acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
poster

A computational model for spatial expression resolution

Published: 12 November 2007 Publication History

Abstract

This paper presents a computational model for the interpretation of linguistic spatial propositions in the restricted realm of a 2D puzzle game. Based on an experiment aimed at analyzing human judgment of spatial expressions, we establish a set of criteria that explain human preference for certain interpretations over others. For each of these criteria, we define a metric that combines the semantic and pragmatic contextual information regarding the game as well as the utterance being resolved. Each metric gives rise to a potential field that characterizes the degree of likelihood for carrying out the instruction at a specific hypothesized location. We resort to machine learning techniques to determine a model of spatial relationships from the data collected during the experiment. Sentence interpretation occurs by matching the potential field of each of its possible interpretations to the model at hand. The system's explanation capabilities lead to the correct assessment of ambiguous situated utterances for a large percentage of the collected expressions.

References

[1]
Bateman, J., Fischer, K., and Tenbrink, T. Why a static interpretation is not sufficient in spatial communication. Proceedings of the EACL Workshop on Dialogue Systems: Interaction, Adaptation and Styles of Management, 2003.
[2]
Baus, J., and Kray, C. Frames of Reference, Positional Information and Navigational Assistance. Proceedings of the 15th International AAAI FLAIRS Conference, pp. 461--465, 2002.
[3]
Bohus, D., and Rudnicky, A. Sorry, I Didn't Catch That! An Investigation of Non-understanding Errors and Recovery Strategies. Proceedings of the 6th SIGdial Workshop on Discourse and Dialogue, Lisbon, Portugal, 2005.
[4]
Brenner, M., Hawes, N., Kelleher, J., and Wyatt, J. Mediating between Qualitative and Quantitative Representations for Task-Oriented Human-Robot Interaction. Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), Hyderabad, India, 2007.
[5]
Byron, D.K., and Stoia, L. An Analysis of Proximity Markers in Collaborative Dialog. Proceedings of the 41st Annual Meeting of the Chicago Linguistic Society, 2005.
[6]
Carbonell, J.G. Discourse pragmatics and ellipsis resolution in task-oriented natural language interfaces. Proceedings of the 21st annual meeting on Association for Computational Linguistics, pp. 164--168, 1983.
[7]
Corradini, A., Hanneforth, T., Bak, A. A Robust Spoken Language Architecture to Control a 2D Game. Proc. of the AAAI International FLAIRS Conference, Key West, FL, USA, pp. 199--204, 2007.
[8]
Costello, F., and Kelleher, J. Spatial prepositions in context: The semantics of near in the presence of distractor objects. Proceedings of the ACL-Sigsem Workshop on Prepositions, 2006.
[9]
Cui, Z., Cohn, A.G., and Randell, D.A. Qualitative Simulation Based on a Logical Formalism of Space and Time. Proceedings of 10th National Conference of the AAAI, 1992.
[10]
Engenhofer, M. J. Reasoning about Binary Topological Relations. Proceedings of the 2nd International Symposium on Advances in Spatial Databases, pp. 143--160, 1991.
[11]
Egenhofer, M. J., and Shariff, A. Metric Details for Natural Language Spatial Relations. ACM Transactions on Information Systems, 16:(4), pp.295--321, 1998.
[12]
Eschenbach, C. Metric Details for Natural Language Spatial Relations. ACM Transactions on Information Systems, 16:(4), pp.295--321, 1999.
[13]
Fernández, R., Corradini, A., Schlangen, D., and Stede, M. Towards Reducing and Managing Uncertainty in Spoken Dialogue Systems. Proceedings of the 7th International Workshop on Computational Semantics, 2007.
[14]
Fischer, K., and Moratz, R. From Communicative Strategies to Cognitive Modeling. Proceedings of the 1st International Workshop on Epigenetic Robotics, 2001.
[15]
Gorniak, P., and Roy, D. Grounded Semantic Composition for Visual Scenes. Journal of Artificial Intelligence Research, Vol. 21, pp. 429--470, 2004.
[16]
Gorniak, P., and Roy, D. Speaking with your Sidekick: Understanding Situated Speech in Computer Role Playing Games. Proceedings of Artificial Intelligence and Digital Entertainment, 2005.
[17]
Gorniak, P., Orkin, J., and Roy, D. Speech, Space and Purpose: Situated Language Understanding in Computer Games. 28th Annual Meeting of the Cognitive Science Society Workshop on Computer Games, 2006.
[18]
Kaiser, E., D., Olwal, McGee, A., Benko, H., Corradini, A., Li, X., Cohen, P.R., and Feiner, S. Mutual Disambiguation of 3D Multimodal Interaction in Augmented and Virtual Reality. Proceedings of the ACM 5th International Conference on Multimodal Interfaces, Vancouver, Canada, BC, pp. 12--19, 2003.
[19]
Kelleher, J., Costello, F., and van Genabith, J. Dynamically structuring, updating and interrelating representations of visual and linguistic discourse context. Artificial Intelligence, Special volume on connecting language to the world, 167(1--2):62--102, 2005.
[20]
Kelleher, J., Kruijff, G. J., and Costello, F. Proximity in Context: an empirically grounded computational model of proximity for processing topological spatial expressions. Proceedings of the ACL COLING, 2006.
[21]
Krahmer, E., and Piwek, P. Presupposition Projection as Proof Construction. In: Bunt, H. & Muskens R. (eds.), Computing Meanings: Current issues in Computational Semantics, Kluwer Academic Publisher, Dordrecht, 1999.
[22]
Kray, C., and Blocher, A. Modeling the Basic Meanings of Path Relations. Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), San Francisco, CA, USA, pp. 384--393, 1999.
[23]
Krüger, A., and Maaß, W. Towards a Computational Semantics of Path Relations. Workshop on Language and Space at the 14th National Conference on Artificial Intelligence, 1997.
[24]
Lemon, O., Bracy, A., Gruenstein, A., and Peters, S. A Multi-Modal Dialogue System for Human-Robot Conversation. Proceedings of NAACL, 2001.
[25]
Levit, M., and Roy, D. Interpretation of Spatial Language in a Map Navigation Task. IEEE Transactions on Systems, Man, and Cybernetics, 2007.
[26]
Logan, D. and Sadler, D. A Computational Analysis of the Apprehension of Spatial Relations. In: Bloom, M. et al. (eds), Language and Space, MIT Press, 1996.
[27]
Moratz, R., Fischer, K., and Tenbrink, T. Cognitive Modeling of Spatial Reference for Human--Robot Interaction. International Journal on Artificial Intelligence Tools, 10(4): 589--611, 2001.
[28]
Poesio, M., Sturt, P., Artstein, R., and Filik, R. Under specification and anaphora: Theoretical issues and preliminary evidence. Discourse Processes, 42(2):157--175, 2006.
[29]
Regier, T., and Carlson, L. Grounding spatial language in perception: An empirical and computational investigation. Journal of Experimental Psychology: General, 130(2):273--298, 2001.
[30]
Roy, D. Gorniak, P., Mukherjee, N., and Juster, J. A Trainable Spoken Language Understanding System for Visual Object Selection. Proceedings of the ICSLP, 2002.
[31]
Roy, D. Learning Visually Grounded Words and Syntax for a Scene Description Task. Computer Speech and Language, 2002.
[32]
Roy, D., and Mukherjee, N. Towards situated speech understanding: Visual Context Priming of Language Models. Computer Speech and Language, 19(2):227--248, 2005.
[33]
Tapus, A., Vasudevan, S., and Siegwart, R. Towards a multilevel cognitive probabilistic representation of space. Proceedings of the SPIE, Volume 5666, pp. 39--48, 2005.
[34]
Webber, B., Stone, M., Joshi, A., and Knott, A. Anaphora and Discourse Structure. Computational Linguistics, 29(4), pp. 545--587, 2003.
[35]
Winograd, T. Procedures as a Representation for Data in a Computer Program for Understanding Natural Language. MIT AI Technical Report 235, 1971.

Cited By

View all
  • (2008)Tailoring the Interpretation of Spatial Utterances for Playing a Board GameProceedings of the 13th international conference on Artificial Intelligence: Methodology, Systems, and Applications10.1007/978-3-540-85776-1_5(45-57)Online publication date: 4-Sep-2008

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICMI '07: Proceedings of the 9th international conference on Multimodal interfaces
November 2007
402 pages
ISBN:9781595938176
DOI:10.1145/1322192
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2007

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. machine learning
  2. psycholinguistic study
  3. spatial expressions

Qualifiers

  • Poster

Conference

ICMI07
Sponsor:
ICMI07: International Conference on Multimodal Interface
November 12 - 15, 2007
Aichi, Nagoya, Japan

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2008)Tailoring the Interpretation of Spatial Utterances for Playing a Board GameProceedings of the 13th international conference on Artificial Intelligence: Methodology, Systems, and Applications10.1007/978-3-540-85776-1_5(45-57)Online publication date: 4-Sep-2008

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media