Skip to main content
Springer Nature Link
Log in

An historical overview of natural language processing systems that learn

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

A fundamental issue in natural language processing is the prerequisite of an enormous quantity of preprogrammed knowledge concerning both the language and the domain under examination. Manual acquisition of this knowledge is tedious and error prone. Development of an automated acquisition process would prove invaluable.

This paper references and overviews a range of the systems that have been developed in the domain of machine learning and natural language processing. Each system is categorised into either a symbolic or connectionist paradigm, and has its own characteristics and limitations described.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Allen, R. B. & Riecken, M. E. (1988). Anaphora and Reference in Connectionist Language Users. In Proceedings ofThe International Computer Science Conference. Hong Kong.

  • Anderson, J. R. (1977). Induction of Augmented Transition Networks.Cognitive Science 1: 125–157.

    Google Scholar 

  • Anderson, J. R. & Bower, G. H. (1973)Human Associative Memory. Winston: Washington D.C.

    Google Scholar 

  • Berwick, R. (1985).The Acquisition of Syntactic Knowledge. MIT Press: Cambridge, Massachusetts.

    Google Scholar 

  • Birnbaum, L. & Selfridge, M. (1979).Problems in the Conceptual Analysis of Natural Language. Research Report 168, Department of Computer Science, Yale University, New Haven, Connecticut.

  • Bybee, J. L. & Slobin, D. I. (1982). Rules and Schemas in the Development and Use of the English Past Tense.Language 58: 265–289.

    Google Scholar 

  • Carbonell, J. G. (1983). An Overview of Machine Learning. In Michalski, R. S., Carbonell, J. G. & Mitchell, T. M. (eds.)Machine Learning: An Artificial Intelligence Approach, 3–23. Tioga: Palo Alto, California.

    Google Scholar 

  • Chomsky, N. (1957).Syntactic Structures. Mouton: The Hague, Holland.

    Google Scholar 

  • Chomsky, N. (1965).Aspects of the Theory of Syntax. MIT Press: Cambridge, Massachusetts.

    Google Scholar 

  • Chomsky, N. (1981). Lectures on Government and Bionding. InSeries of Studies in Generative Grammar 9, Foris Publications: Dordrecht, The Netherlands.

    Google Scholar 

  • Collier, R. (1993). Knowledge Acquisition from Technical Texts Using Natural Language Processing Techniques. In Proceedings ofThe Second Workshop on the Cognitive Science of Natural Language Processing. Dublin, Eire: Dublin City University.

    Google Scholar 

  • Dejong, G. F. (1983). Acquiring Schemata Through Understanding and Generalising plans. In Proceedings ofThe Eighth International Joint Conference on Artificial Intelligence. Karlsruhe, West Germany: Morgan Kaufmann.

    Google Scholar 

  • Fodor, J. A. & Pylyshyn, S. W. (1988). Connectionism and Cognitive Architecture: A Critical Analysis.Cognition 28: 2–71.

    Google Scholar 

  • Francis, W. N. & Kucera, H. (1979).Manual of Information to Accompany a Standard Corpus of Present-Day Edited American English for Use with Digital Computers. Technical Report, Department of Linguistics, Brown University, Rhode Island.

    Google Scholar 

  • Garvin, P. I. (1967). The Automation of Discovery Procedure in Linguistics.Lanuage 43: 172–178.

    Google Scholar 

  • Gasser, M. E. (1988).A Connectionist Model of Sequences Generation in a First and Second Language. Technical Report, UCLA-AI-88-13, Artificial Intelligence Laboratory, Computer Science Department, University of California, Los Angeles.

    Google Scholar 

  • Gold, E. (1967). Language Identification in the Limit.Information and Control 16: 447–474.

    Google Scholar 

  • Granger, R. H. (1977). FOUL-UP: A Program that Figures out Meanings of Words from Context. In Proceedings ofThe Fifth International Joint Conference on Artificial Intelligence, 172–178. Cambridge, Massachusetts: Morgan Kaufmann.

    Google Scholar 

  • Hanson, S. J. & Kegl, J. (1987). PARSNIP: A Connectionist Network that Learns Natural Language Grammar from Exposure to Natural Language Sentences. In Proceedings ofThe Ninth Annual Conference of the Cognitive Science Society, 106–119. Seattle, Washington: Lawrence Erlbaum.

    Google Scholar 

  • Hedrick, C. (1976). Learning Production Systems from Examples.Artificial Intelligence 7: 21–49.

    Google Scholar 

  • Hill, J. C. (1983). A Computational Model of Language Acquisition of the Two Year Old.Cognition and Brain Theory 6(3): 287–317.

    Google Scholar 

  • Hinton, G. E. (1981). Implementing Semantic Networks in Parallel Hardware. In Hinton, G. E. & Anderson, J. A. (eds.)Parallel Models of Associative Memory, Lawrence Erlbaum: Hillsdale, New Jersey.

    Google Scholar 

  • Jackendoff, R. (1983).Semantics and Cognition. MIT Press: Cambridge, Massachusetts, and London, England.

    Google Scholar 

  • Kelley, K. L. (1967).Early Syntactic Acquisition. Technical Report Number P-3179, Rand Corporation, Santa Monica, California.

    Google Scholar 

  • Kolodner, J. L. (1980).Retrieval and Organizational Strategies in Conceptual Memory: A Computer Model, Technical Report 187, Department of Computer Science, Yale University, New Haven, Connecticut.

    Google Scholar 

  • Knowlton, K. (1962).Sentence Parsing with a Self-Organising Heuristic Program. Ph.D. Dissertation, Massachusetts Institute of Technology, Massachusetts.

    Google Scholar 

  • Kuczaj, S. A. (1977). The Acquisition of Regular and Irregular Past Tense Forms.Journal of Verbal Learning and Verbal Behaviour 16: 589–600.

    Google Scholar 

  • Langley, P. (1982). Language Acquisition Through Error Recovery.Cognition and Brain Theory 5(3), 211–255.

    Google Scholar 

  • Langley, P. & Neches, R. T. (1981).PRISM User's Manual. Technical Report, Department of Computer Science, Carnegie-Mellon University, Pennsylvania.

    Google Scholar 

  • Lebowitz, M. (1980).Generalisation and Memory in an Integrate Understanding System. Technical Report 186, Yale University, Dept. of Computer Science, New Haven, Connecticut.

    Google Scholar 

  • Lebowitz, M. (1983). Generalisation from Natural Language Text.Cognitive Science 7: 1–40.

    Google Scholar 

  • MacWhinney, B. (1987). The Competition Model. In MacWhinney, B. (ed.)Mechanisms of Language Acquisition, 249–308. Lawrence Erlbaum: Hillsdale, NJ.

    Google Scholar 

  • McKevitt, P. (1994). Special Issue on the Integration of Natural Language and Vision Processing.Artificial Intelligence Review (special volume) issues 1, 2, 3. Kluwer Academic Publisher: Dordrecht, The Netherlands (forthcoming).

    Google Scholar 

  • McKevitt, P., Partridge, D. & Wilks, Y. (1992). Approaches to Natural Language Discourse Processing.Artificial Intelligence Review 6(4): 333–364.

    Google Scholar 

  • Michalski, R. (1992). Understanding the Nature of Learning: Issues and Research Directions. In Michalski, R. S., Carbonell, J. G. & Mitchell, T. M. (eds.)Machine Learning, an Artificial Intelligence Approach 2: 3–25. Morgan Kaufmann: Los Altos, California.

    Google Scholar 

  • Minsky, M. (1975). A Framework for Representing Knowledge. In Winston, P. H. (ed.)The Psychology of Computer Vision, 211–217. McGraw-Hill: New York.

    Google Scholar 

  • Mooney, R. (1985).Generalising Explanations of Narratives into Schemata. Technical Report T-147, Coordinated Science Laboratory, University of Illinois, Urbana.

    Google Scholar 

  • O'Rorke, P. (1984). Generalisation of Explanation-Based Schema Acquisition. In Proceedings ofThe National Conference on Artificial Intelligence. Austin, Texas: AAAI.

    Google Scholar 

  • Pinker, S. (1979). Formal Models of Language Learning.Cognition 7: 217–283.

    Google Scholar 

  • Pinker, S & Prince, A. (1988). On Language and Connectionism: Analysis of a Parallel Distributed Processing Model of Language Acquisition.Cognition 28: 73–193.

    Google Scholar 

  • Pollack, J. B. (1990). Recursive Distributed Representations.Artificial Intelligence 46: 77–105.

    Google Scholar 

  • Powers, D. M. (1990).Goals, Issues and Directions in Machine Learning of Natural Language and Ontology. Information additional to Call for participation, AAAI Spring Symposium, Stanford, California.

    Google Scholar 

  • Reeker, L. H. (1975).An Examination of Innateness Arguments in Language Acquisition. Technical Report Number TR-75-4, Department of Computer Science, University of Oregon, Eugene.

    Google Scholar 

  • Reeker, L. H. (1976). The Computational Study of Language Acquisition.Advances in Computers 15: 181–237.

    Google Scholar 

  • Rumelhart, D. E., Hinton, G. E. & Williams, R. (1986). Learning Internal Representations Through Error Propagation. In Rumelhart, D. E., McClelland, J. L. and the PDP Research Group (eds.)Parallel Distributed Processing: Experiments in the Microstructure of Cognition 1: Foundations, 22–40. MIT Press: Cambridge, Massachusetts.

    Google Scholar 

  • Rumelhart, D. E. & McClelland, J. L. (1986a). On Learning the Past Tenses of English Verbs. In McClelland, J. & Rumelhart, D. (eds.)Parallel Distributed Processing, Volume 2: Psychological and Biological Models, 216–271. MIT Press: Cambridge, Massachusetts.

    Google Scholar 

  • Rumelhart, D. E. & McClelland, J. L. (eds.) (1986b).Parallel Distributed Processing, Volumes 1 and 2. MIT Press, Cambridge, Massachusetts.

    Google Scholar 

  • Salveter, S. C. (1979). Inferring Conceptual Graphs.Cognitive Science 3(2): 141–166.

    Google Scholar 

  • Schank, R. C. (1969).A Conceptual Dependency Representation for a Computer Oriented Semantics. Artificial Intelligence Memo Number 172, Computer Science Department, Stanford University, Stanford, California.

    Google Scholar 

  • Schank, R. C.et al. (1975).SAM — A Story Understander. Research Report #43, Department of Computer Science, Yale University, New Haven, Connecticut.

    Google Scholar 

  • Schank, R. C. & Abelson, R. (1977).Scripts, Plans, Goals and Understanding. Lawrence Erlbaum: Hillsdale, New Jersey.

    Google Scholar 

  • Segre, A. M. & Dejong, G. F. (1985). Explanation Based Manipulator Learning: Acquisition of Planning Ability Through Observation. In Proceedings ofThe IEEE International Conference on Robotics and Automation, 1031–1038. St. Louis, Missouri: IEEE.

    Google Scholar 

  • Selfridge, M. (1980).A Process Model of Language Acquisition. Computer Science Technical Report 172, Yale University, New Haven, Connecticut.

    Google Scholar 

  • Selfridge, M. (1981). A Computer Model of Child Language Acquisition. In Proceedings ofThe Seventh International Joint Conference on Artificial Intelligence, 446–451. Vancouver, Canada: Morgan Kaufmann.

    Google Scholar 

  • Shapiro, S. C. (Editor-in-chief) (1992).Encyclopedia of Artificial Intelligence, Second Edition. John Wiley & Sons Inc.: New York.

    Google Scholar 

  • Sharkey, N. E. (1988). A PDP System for Goal-Plan Decision. In Trappl, R. (ed.)Cybernetics and Systems, 1031–1038. Kluwer Academic: Dordrecht, The Netherlands.

    Google Scholar 

  • Shavlik, J. (1985). Learning about Momentum Conservation. In Proceedings ofThe Ninth International Joint Conference on Artificial Intelligence. Los Angeles, California: AAAI.

    Google Scholar 

  • Siklossy, L. (1972). Natural Language Learning by Computer. In Simon, H. A. & Siklossy, L. (eds.)Representation and Meaning: Experiments with Information Procession Systems. Prentice-Hall: Englewood Cliffs, New Jersey.

    Google Scholar 

  • Siskind, J. M. (1990). Acquiring Core Meanings of Words, Represented as Jackendoff-Style Conceptual Structures, from Correlated Streams of Linguistic and Non-Linguistic Input. In Proceedings ofThe Twenty-eighth Annual Meeting of the Association for Computational Linguistics, 143–156. Pennsylvania: Association for Computational Linguistics, University of Pittsburgh.

    Google Scholar 

  • Small, S. L., Cottrell, G. W. & Shastri, L. (1982). Towards Connectionist Parsing. In Proceedings ofThe National Conference on Artificial Intelligence. Pittsburgh, Pennsylvania: AAAI.

    Google Scholar 

  • Solomonoff, R. (1959). A New Method for Discovering the Grammars of Phrase Structure Languages. In Proceedings ofThe International Conference on Information Processing. United Nations Educational, Scientific and Cultural Organisation.

  • Waltz, D. L. & Pollack, J. B. (1985). Massively Parallel Parsing: A Strongly Interactive Model of Natural Language Interpretation.Cognitive Science 9: 51–74.

    Google Scholar 

  • Wickelgren, W. A. (1969). Context-Sensitive Coding, Associative Memory, and Serial Order in (Speech) Behaviour.Psychological Review 76: 1–15.

    Google Scholar 

  • Wilensky, R. (1983).Planning and Understanding: A Computational Approach to Human Reasoning. Addison-Wesley Advanced Book Program. Addison-Wesley: Reading, Massachusetts.

    Google Scholar 

  • Winston, P. H. (1970).Learning Structural Descriptions from Examples. Artificial Intelligence Technical Report-231, MIT, Artificial Intelligence Laboratory, Cambridge, Massachusetts.

    Google Scholar 

  • Wilks, Y. (1973).Preference Semantics, Memo AIM-206, Artificial Intelligence Laboratory, Stanford University, Stanford, California.

    Google Scholar 

  • Woods, W. A. (1970). An Experimental Parsing System for Transition Network Grammars. In Rustin, R. (ed.)Natural Language Processing. Prentice Hall: Englewood Cliffs, New Jersey.

    Google Scholar 

  • Zernik, U. & Dyer, M. G. (1986). Disambiguation and Acquisition Through the Phrasal Lexicon. In Proceedings ofThe 11th International Conference on Computational Linguistics. Bonn, Germany: Association for Computational Linguistics.

    Google Scholar 

  • Zernik, U. (1987). Language Acquisition: Learning a Hierarchy of Phrases. In Proceedings ofThe Tenth International Joint Conference on Artificial Intelligence, 125–132. Milan, Italy: Morgan Kaufmann.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Regent Court, University of Sheffield, 211 Portobello Street, Sheffield, England

    Robin Collier

Authors
  1. Robin Collier
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Collier, R. An historical overview of natural language processing systems that learn. Artif Intell Rev 8, 17–54 (1994). https://doi.org/10.1007/BF00851349

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00851349

Key words