Skip to main content
SpringerLink
Log in

Processing Descriptions containing Words and Gestures — A System Architecture

  • Conference paper
GWAI-86 und 2. Österreichische Artificial-Intelligence-Tagung

Part of the book series: Informatik-Fachberichte ((2252,volume 124))

Abstract

In person-to-person dialog, a lot of pointing gestures are used in order to facilitate and speed up the interaction. Our aim is to integrate this combination of NL and deixis into man-machine communication.

Since the analysis of pointing gestures is highly interwoven with the analysis of noun phrases, a system has been developed which accepts mouse dicks as part of a noun phrase in NL sentences.

The syntactically and morphologically processed input sentence is represented as a functional-semantic structure by the intrasentential analysis. Aided by specific translation rules, the referential-semantic interpretation evaluates this structure with respect to three types of knowledge: the region concurrently pointed at on the screen, the relation to the dialog memory, and the associated indvidalized part of the conceptual knowledge base. This referent-identification process results in determining the intended referential object.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Allgayer, J. (1986): Eine Graphikkomponente zur Integration von Zeigehandlungen in natürlich-sprachliche KI-Systeme. In: GI-Jahrestagung 1986. Berlin: Springer.

    Google Scholar 

  • Bobrow, R.J., Webber, B.L. (1980): Knowledge Representation for Syntactic/Semantic Processing. Proceedings of AAAI80.

    Google Scholar 

  • Brachman, R.J., Schmolze, J.G. (1985): An Overview of the KL-ONE Knowledge Representation System. Cognitive Science 9.

    Google Scholar 

  • Bühler, K. (1982): The Deictic Field of Language and Deictic Words. Abridged translation of K. Bühler (1934): Sprachtheorie, part 2, chapters 7 and 8. In: R.J. Jarvella and W. Klein. eds.: Speech, Place, and Action. Chicester etc.: Wiley.

    Google Scholar 

  • Cohen, P.R. (1984): The Pragmatics of Referring and the Modality of Communication. Computational Linguistics Vol. 10, No. 2.

    Google Scholar 

  • Finkler, W., Neumann, G. (1986): MORPHIX - ein hochportabler Lemmatisierungsmodul filr das Deutsche. Final Report of the advanced practica, run by Dipl.-Inform. N. Reithinger, WS ‘85/86. Fachbereich Informatik, Universität des Saarlandes.

    Google Scholar 

  • Habel, Chr. (1984): Zur Repräsentation der referentiellen Struktur. In: Rollinger, C.-R. (Hrsg.): Probleme des (Text-)Verstehens - Ansätze der Künstlichen Intelligenz. Tübingen: Niemeyer.

    Google Scholar 

  • v. Hahn, W.,Hoeppner, W. Jameson, A., Wahlster, W. (1979): The Anatomy of the Natural Language Dialog System HAM-RPM. Mai 1979. In: Bolc, L. (ed): Natural Language Based Computer Systems. München: Hanser/Macmillan 1980, 119–253

    Google Scholar 

  • Karttunen, L. (1986): D-PATR. A Development Environment for Unification-Based Grammars. SRI International and Center for the Study of Language and Information, Stanford.

    Google Scholar 

  • Schmauks, D.: Formulardeixis und ihre Simulation auf dem Bildschirm. Ein Überblick aus linguistischer Sicht. Memo No. 4, Sonderforschungsbereich 314, Dept. of Computer Science, University of Saarbrücken, West Germany

    Google Scholar 

  • Schwarz, D.S. (1979): Naming and Referring. The Semantics and Pragmatics of Singular Terms. Berlin: de Gruyter.

    Google Scholar 

  • Shieber, S., H. Uszkoreit, F. Pereira, J. Robinson, and M. Tyson, 1983: “The Formalism and Implementation of PATR-II”, in B. Grosz and M. Stickel, Research on Interactive Acquisition and Use of Knowledge, SRI Final Report 1984, SRI International, Menlo Park, California.

    Google Scholar 

  • Sondheimer, N.K., Weischedel, R.M., Bobrow, R.J. (1984): Semantic Interpretation Using KLONE. Proceedings of Coling84. Stanford University, California.

    Google Scholar 

  • Winograd, T. (1983): Language as a Cognitive Process. Volume I: Syntax. Reading, Mass.: Addison-Wesley.

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. SFB 314: AI — Knowledge-Based Systems FB 10 — Department of Computer Science, University of Saarbrücken, D-6600, Saarbrücken 11, West Germany

    Jürgen Allgayer & Carola Reddig

Authors
  1. Jürgen Allgayer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carola Reddig
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. IBM Deutschland GmbH, Abt. LILOG 3504, Postfach 800880, D-7000, Stuttgart 80, Germany

    Claus-Rainer Rollinger

  2. Institut für Medizinische Kybernetik und Artificial Intelligence, Universität Wien, Freyung 6, A-1010, Wien, Austria

    Werner Horn

Rights and permissions

Reprints and permissions

Copyright information

© 1986 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Allgayer, J., Reddig, C. (1986). Processing Descriptions containing Words and Gestures — A System Architecture. In: Rollinger, CR., Horn, W. (eds) GWAI-86 und 2. Österreichische Artificial-Intelligence-Tagung. Informatik-Fachberichte, vol 124. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-71385-9_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-71385-9_12

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-16808-9

  • Online ISBN: 978-3-642-71385-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation