Abstract
We will discuss various connectionist schemes for natural language understanding (NLU). In principle, massively parallel processing schemes, such as connectionist networks, are well-suited for modelling highly integrated forms of processing. The connectionist approach towards natural language processing is motivated by the belief that a NLU system should process knowledge from many different sources, e.g. semantic, syntactic, and pragmatic, in just this sort of integrated manner. The successful use of spreading activation for various disambiguation tasks in natural language processing models lead to the first connectionist NLU systems. In addition to describing in detail a connectionist disambiguation system, we will also discuss proposed connectionist approaches towards parsing and case role assignment. This paper is intended to introduce the reader to some of the basic ideas behind the connectionist approach to NLU. We will also suggest some directions for future research.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Charniak, E. (1983) Passing markers: A theory of contextual influence in language comprehension. Cognitive Science, 7, July–September 1983, 171–190.
Charniak, E. & Santos, E. (1987). A connectionist context-free parser which is not context-free, but then it is not really connectionist either. Proceedings of the Ninth Annual Conference of the Cognitive Science Society, Seattle, WA, July 1987, 70–77.
Collins, A.M. & Loftus, E.F. (1975). A spreading-activation theory of semantic processing. Psychological Review, November 1975. 82, 407–429.
Conttrell, G.W. (1985). A connectionist approach to word sense disambiguation. Doctoral dissertation, Computer Science Department, University of Rochester, Rochester, NY 14627, April 1985.
Cottrell, G.W. & Small, S.L. (1983). A connectionist scheme for modelling word sense disambiguation. Cognition and Brain Theory, 6(1), 1983, 89–120.
Fahlman, S.E. (1979). NETL: A system for representing and using real-world knowledge. MIT Press. Cambridge, Massachusetts, USA.
Fahlman, S.E., Hinton, G.E. & Sejnowski, T.J. (1983). Massively parallel architectures for AI: NETL, Thistle and Boltzmann machines. Proceedings of the National Conference on Artificial Intelligence, Washington, August 1983. 109–113.
Fanty, M. (1985). Context-Free Parsing in Connectionist Networks. Technical Report 174, Computer Science Department, University of Rochester, Rochester NY, November 1985.
Feldman, J.A. (1985). Energy and the Behavior of Connectionist Models. Technical Report 155, Computer Science Department, University of Rochester, Rochester NY, November 1985.
Feldman, J.A. & Ballard, D.H. (1982). Connectionist models and their properties. Cognitive Science, 6, 205–254.
Gentner, D. (1982). Some interesting differences between nouns and verbs. Cognition and Brain Theory, 4, 155–184.
Hinton, G.E. (1981). Implementing semantic networks in parallel hardware. In Parallel Models of Associative Memory, (eds.) G.E.Hinton and J.A.Anderson. Erlbaum, Hillsdale, NJ, U.S.A.
Hinton, G.E. (1988). Representing part-whole hierarchies in connectionist networks. Proceedings of the Tenth Annual Conference of the Cognitive Science Society, Montreal, Canada, 48–54.
Hirst, G. (1983), Semantic interpretation against ambiguity. PhD thesis, Department of Computer Science of the Brown University, December 1983. Appeared as Semantic interpretation and the resolution of ambiguity (Studies in natural language processing). Cambridge University Press, 1987.
Kirkpatrick, S., Gelatt, C.D.Jr & Vecchi, M.P. (1983). Optimization by simulated annealing, Science, 220, 4598, 671–680.
McClelland, J.L. & Kawamoto, A.H. (1986). Mechanisms of sentence processing: assigning roles to constituents of sentences. In Parallel Distributed Processing by D.E. Rumelhart, J.L. McClelland and the PDP Research group, vol. 2, Bradford/MIT Press, Cambridge, USA, 273–325.
Pollack, J.B. (1987). Cascade back-propagation on dynamic connectionist networks. Proceedings of the Ninth Annual Conference of the Cognitive Science Society, Seattle, WA, July 1987, 391–404.
Rosenblatt, F. (1962). Principles of Neurodynamics. Spartan, New York.
Sampson, G. (1986). A stochastic approach to parsing. Proceedings of Coling '86, 1986, 151–155.
Schank, R.C. (1975). Conceptual Information Processing. North-Holland publishing company, Amsterdam.
Selman, B. (1985). Rule-based Processing in a Connectionist System for Natural Language Understanding. Technical Report CSRI-168, Computer Systems Research Group, University of Toronto, April 1985.
Selman, B. & Hirst, G. (1985). A Rule-Based Connectionist Parsing System, Proceedings of the Seventh Annual Conference of the Cognitive Science Society, Irvine, CA, August 1985, 212–219. An extended version entitled ‘Parsing as an Energy Minimization Problem’ appeared in Genetic Algorithms and Simulated Annealing (ed.) Lawrence Davis, Pitman, London. 155–168.
Small, S.L., Cottrell, C. & Shastri, L. (1982). Towards Connectionist Parsing. Proceedings of the National Conference on Artificial Intelligence, Pittsburgh, PA, August 1982, 247–250.
Waltz, D.L. & Pollack, J.B. (1985). Massively parallel parsing. Cognitive Science, 9, 1985 51–74.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Selman, B. Connectionist systems for natural language understanding. Artif Intell Rev 3, 23–31 (1989). https://doi.org/10.1007/BF00139194
Issue Date:
DOI: https://doi.org/10.1007/BF00139194