Abstract
I will present WordsEye, a natural language understanding system that generates three-dimensional scenes from English descriptions of those scenes (joint work with Bob Coyne, Chris Johnson, Owen Rambow, Srinivas Bangalore).
WordsEye works by first parsing the input, using Church’s (1988) part of speech tagger and a version of Michael Collins’ (1999) parser. The parser output is converted into a dependency structure, and semantic rules are then applied to produce a high-level scene description. This scene description is then fed into a set of depiction rules that decide what objects to use, how to arrange them, and in the case of human figures, how to pose them depending upon the action being depicted. WordsEye also does reference resolution, which is obviously crucial for deciding whether a just-named object is in fact the same as an object previously named in the discourse.
The talk will give an overview of the current state of WordsEye, detailing the natural language processing component and the depiction component. I will show how WordsEye depicts spatial relations, actions, attributes, properties of the environment and how it does rudimentary common-sense reasoning to resolve potential conflicts in depiction. The talk will also discuss possible applications of this technology, as well as future directions for research in this area.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sproat, R. (2002). WordsEye: A Text-to-Scene Conversion System. In: Ranchhod, E., Mamede, N.J. (eds) Advances in Natural Language Processing. PorTAL 2002. Lecture Notes in Computer Science(), vol 2389. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45433-0_1
Download citation
DOI: https://doi.org/10.1007/3-540-45433-0_1
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43829-8
Online ISBN: 978-3-540-45433-5
eBook Packages: Springer Book Archive