Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Natural Language Generation

IWNLG 1992: Aspects of Automated Natural Language Generation pp 247–262Cite as

Generating referring expressions in a multimodal environment

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 587))

Abstract

EDWARD is a system which is being developed to study multimodal human-computer interaction. It incorporates a graph-editor called Gr2 and a Dutch natural language dialogue system called DoNaLD. EDWARD is capable of realizing referring actions in three ways: it can utter unimodal referring expressions, it can generate pointing gestures and it can produce multimodal referring expressions which combine referring expressions with a pointing gesture. The system uses its knowledge base and a context model to decide the type and the conceptual content of its referring expressions. The context model used is based on Alshawi's notions of context factors and salience. Presently seven types of context factors are used. The decision tree and a set of rules used by EDWARD to guide the generation process are described.

This research was carried out within the framework of the research programme ‘Human-Computer Communication using natural language’ (MMC). The MMC-programme is sponsored by SPIN Stimuleringsprojectteam Informaticaonderzoek, BSO, Digital Equipment B.V.

This is a preview of subscription content, log in via an institution.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Allgayer, J., Jansen-Winkeln, R., Reddig, C., Reithinger, N.: Bidirectional use of knowledge in the multi-modal NL access system XTRA. Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, Detroit, MI USA. 20–25 August 1989 (pp. 1492–1497).

    Google Scholar 

  • Alshawi, H. (1987): Memory and Context for Language Interpretation. Cambridge (UK): Cambridge University Press.

    Google Scholar 

  • Bos, E. (in press): A Graph-Editor. In L. Neal and G. Szwillus (Eds.). Syntax-Directed Editing. New York: Academic Press.

    Google Scholar 

  • Brachman, R., Schmolze, J. (1985): An overview of the KL-ONE knowledge representation system. Cognitive Science, 9, 171–216.

    Google Scholar 

  • Claassen, W., Huls, C. (1990): DoNaLD: A Dutch Natural Language Dialogue system (SPIN/MMC Research Report no. 11). Nijmegen, The Netherlands: NICI.

    Google Scholar 

  • Claassen, W., Bos, E., Huls, C. (1990): The Pooh Way in Human-Computer Interaction: Towards Multimodal Interfaces (SPIN/MMC Research Report no. 5). Nijmegen, The Netherlands: NICI.

    Google Scholar 

  • Dale, R., Haddock, N.: Generating Referring Expressions Involving Relations. Proceedings of the Fifth Meeting of the European Chapter of the Association for Computational Linguistics, Berlin, Germany, April 1991 (pp. 161–166).

    Google Scholar 

  • De Smedt, K., Geurts, B., Desain, P.: Waiting for the gift of sound and vision: On naturallanguage sentence production in multimodal interfaces. ESPRIT Workshop on Natural Language Processing, Brussels, October 1987.

    Google Scholar 

  • Grosz, B.J. (1978): Discourse Knowledge. In D. Walker (Ed.), Understanding Spoken Language. New York: North-Holland.

    Google Scholar 

  • Grosz, B.J., Sidner, C.L. (1986): Attention, Intentions, and the Structure of Discourse. Computational Linguistics, 12(3):175–204, 1986.

    Google Scholar 

  • Neal, J.G., Shapiro, S.C.: Intelligent Multi-media Interface Technology. Proceedings of the workshop on Architectures for Intelligent Interfaces: Elements and Prototypes, Lockhead AI Center, Monterrey, CA, 1988 (pp. 69–91).

    Google Scholar 

  • Pattabhiraman, T., Cercone, N.: Salience in Natural Language Generation. Proceedings of the IJCAI-91 Workshop on Decision Making throughout the Generation Process, Sydney, Australia, August 1991 (pp. 34–41).

    Google Scholar 

  • Reithinger, N.: The performance of an Incremental Generation Component for Multi-modal Dialogue Contributions. In: ????? (1992) ??????? Springer. These Proceedings.

    Google Scholar 

  • Sidner, C.L. (1979): Towards a Computational Theory of Definite Anaphora Comprehension in English Discourse. Ph.D. Thesis, MIT, Cambridge, MA.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. NICI, University of Nijmegen, Montessorilaan 3, Postbus 9104, 6500, HE Nijmegen, The Netherlands

    Wim Claassen

Authors
  1. Wim Claassen
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

R. Dale E. Hovy D. Rösner O. Stock

Rights and permissions

Reprints and permissions

Copyright information

© 1992 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Claassen, W. (1992). Generating referring expressions in a multimodal environment. In: Dale, R., Hovy, E., Rösner, D., Stock, O. (eds) Aspects of Automated Natural Language Generation. IWNLG 1992. Lecture Notes in Computer Science, vol 587. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-55399-1_17

Download citation

  • DOI: https://doi.org/10.1007/3-540-55399-1_17

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-55399-1

  • Online ISBN: 978-3-540-47054-0

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation