Abstract
Research efforts to study computation and cognitive modeling on neurally-inspired mechanisms have come to be called Connectionism. Rather than being brand new, it is actually the rebirth of a research programme which thrived from the 40s through the 60s and then was severely retrenched in the 70s. Connectionism is often posed as a paradigmatic competitor to the Symbolic Processing tradition of Artificial Intelligence (Dreyfus & Dreyfus, 1988), and, indeed, the counterpoint in the timing of their intellectual and commercial fortunes may lead one to believe that research in cognition is merely a zero-sum game. This paper surveys the history of the field, often in relation to AI, discusses its current successes and failures, and makes some predictions for where it might lead in the future.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ackley, D.H., Hinton, G.E. & Sejnowski, T.J. (1985) A learning algorithm for Boltzmann Machines. Cognitive Science, 9, 147–169.
Allen, R. (1987). Several studies on natural language and back propagation. In Institute of Electrical and Electronics Engineers First International Conference on Neural Networks. San Diego, Vol. II, 335–342.
Anderson, J.A., Silverstein, J.W., Ritz, S.A. & Jones, R.S. (1977) Distinctive Features, Categorical Perception, and Probability Learning: Some Applications of a Neural Model. Psychological Review, 84, 413–451.
Ashby, W.R. (1960). Design for a Brain: The origin of adaptive behaviour. (2nd edn). John Wiley & Sons, New York.
Barto, A.G., Anderson, C.W. & Sutton, R.S. (1982) Synthesis of Nonlinear Control Surfaces by a layered Associative Search Network. Biological Cybernetics, 43, 175–185.
Barto, A.G. (1985) Learning by statistical cooperation of self-interested neuron-like computing elements. Human Neurobiology, 4, 229–256.
Cater, J.P. (1987). Successfully using peak learning rates of 10 (and greater) in back-propagation networks with the heuristic learning algorithm. In Institute of Electrical and Electronics Engineers First International Conference on Neural Networks. San Diego, II-645-652.
Cottrell, G.W. (1985a) Connectionist Parsing. In Proceedings of the Seventh Annual Conference of the Cognitive Science Society. Irvine, CA, USA.
Cottrell, G.W. (1985b) A Connectionist Approach to Word-Sense Disambiguation. TR154 Computer Science Department, University of Rochester, Rochester, USA.
Crutchfield, J.P., Farmer, J.D., Packard, N.H. & Shaw, R.S. (1986) Chaos. Scientific American, 255, 46–57.
Cun, Y. Le (1985) A Learning Scheme for Asymmetric Threshold Networks. In Proceedings of Cognitiva 85. Paris, 599–604.
Dahl, E.D. (1987) Accelerated learning using the generalized delta rule. In Institute of Electrical and Electronics Engineers First International Conference on Neural Networks. San Diego, II-523-530.
Dreyfus, H.L. & Dreyfus, S.E. (1988) Making a Mind versus Modeling the Brain: Artificial Intelligence Again at the Crossroads. Daedalus, 117, 15–43.
Dyer, M.G.,(1983). In Depth Understanding. MIT Press, Cambridge, USA.
Dyer, M.G., Flowers, M. & Wang, Y.A. (1988) Weight Matrix = Pattern of Activation: Encoding Semantic Networks as Distributed Representations in DUAL, a PDP architecture. UCLA-AI-88–5, Los Angeles: Artificial Intelligence Laboratory, UCLA.
Elman, J. & Stork, D. (1987) Session on Speech Recognition and Synthesis. In Institute of Electrical and Electronics Engineers First International Conference on Neural Networks. San Diego, IV-381-504.
Elman, J.L. (1988) Finding Structure in Time. Report 8801, San Diego: Center for Research in Language, UCSD.
Fanty, M. (1985) Context-free parsing in Connectionist Networks. TR174, Rochester, N.Y.: Computer Science Department, University of Rochester, USA.
Feldman, J.A. & Ballard, D.H. (1982) Connectionist models and their properties. Cognitive Science, 6, 205–254.
Fodor, J. & Pylyshyn, A. (1988) Connectionism and Cognitive Architecture: A Critical Analysis. Cognition, 28, 3–71.
Gasser, M. & Dyer, M.G. (1988) Sequencing in a Connectionist Model of Language Processing. In Proceedings of the 12th International Conference on Computational Linguistics. Budapest.
Gleick, J. (1987) Chaos: Making a new science. Viking, New York.
Gorman, R.P. & Sejnowski, T.J. (1988) Analysis of hidden units in a layered network trained to classify sonar targets. Neural Networks, 1, 75–90.
Grebogi, C., Ott, E. & Yorke, J.A. (1987) Chaos, Strange Attractors, and Fractal Basin Boundaries in Nonlinear Dynamics, Science, 238, 632–638.
Grossberg, S. (1987) Competitive Learning: From Interactive Activation to Adaptive Resonance. Cognitive Science, 11, 23–63.
Hanson, S.J. & Kegl, J. (1987) PARSNIP: A connectionist network that learns natural language grammar from exposure to natural language sentences. In Proceedings of the Ninth Conference of the Cognitive Science Society. Seattle, 106–119.
Hebb, D.O. (1949) The Organization of Behavior: A Neuropsychological Theory. John Wiley & Sons, New York.
Hinton, G.E. (1986) Learning Distributed Representations of Concepts. In Proceedings of the Eighth Annual Conference of the Cognitive Science Society. Amherst, MA, 1–12.
Hinton, G.E. (1988) Representing part-whole hierarchies in connectionist networks. In Proceedings of the Tenth Annual Conference of the Cognitive Science Society. Montreal, p. 48–54.
Hopfield, J.J. (1982) Neural Networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences USA, 79, 2554–2558.
Hopfield, J.J. & Tank, D.W. (1985) ‘Neural’ computation of decisions in optimization problems. Biological Cybernetics, 52, 141–152.
Huberman, B.A. & Hogg, T. (1987) Phase Transitions in Artificial Intelligence Systems. Artificial Intelligence, 33, 155–172.
Jordan, M.I. (1986) Serial Order: A Parallel Distributed Processing Appraoch. ICS Report 8608, La Jolla: Institute for Cognitive Science, UCSD.
Kawamoto, A.H. (1985) Dynamic Processes in the (Re)Solution of Lexical Ambiguity. Doctoral Dissertation, Providence: Brown University, Department of Psychology, USA.
Kirkpatrick, S., Gelatt, C.D. & Vecchi, M.P. (1983) Optimization by simulated annealing, Science, 220, 671–680.
Klopf, A.H. (1982) The Hedonistic Neuron. Hemisphere Publishing Corporation, Washington, DC.
Kohonen, T. (1977) Associative Memory: A Systems-Theoretical Approach. Springer-Verlag, Berlin.
Kohonen, T., Oja, E. & Lehtio, P. (1981) Storage and Processing of Information in Distributed Associative Memory Systems. In G.E.Hinton & J.A.Anderson, (Eds.), Parallel models of associative memory. Lawrence Erlbaum Associates, Hillsdale, USA.
Kurten, K.E. (1987) Phase transitions in quasirandom neural networks. In Institute of Electrical and Electronics Engineers First International Conference on Neural Networks. San Diego, II-197-204.
Lapedes, A.S. & Farber, R.M. Nonlinear signal processing using several networks: prediction and system modeling. Biological Cybernetics (in press).
Lenat, D.B. (1977) The Ubiquity of Discovery. In Proceedings of the Fifth International Joint Conference on Artifical Intelligence. Cambridge, MA, 1093–1105.
Mandelbrot, B. (1982) The Fractal Geometry of Nature. Freeman, San Francisco.
McClelland, J.L. & Rumelhart, D.E. (1981) An interactive activation model of the effect of context in perception: Part 1. An account of basic findings. Psychology Review, 88, 375–407.
McClelland, J. & Kawamoto, A. (1986) Mechanisms of Sentence Processing: Assigning Roles to Constituents. In J.L.McClelland, D.E.Rumelhart & the PDP research Group, (Eds.), Parallel Distributed Processing: Experiments in the Microstructure of Cognition, Vol. 2. MIT Press, Cambridge, USA.
McCorduck, P. (1979) Machines Who Think. Freeman, San Francisco, CA, USA.
McCulloch, W.S. & Pitts, W. (1943) A logical calculus of the ideas immanent in nervious activity. Bulletin of Mathematical Biophysics, 5, 115–133.
Minsky, M. & Papert, S. (1969) Perceptrons. MIT Press, Cambridge, MA., USA.
Minsky, M. (1986) The Society of Mind. Simon & Schuster, New York.
Norman, D.A. & Rumelhart, D.E. (1975) Explorations in Cognition. W.H. Freeman & Co., San Francisco.
Parker, D.B. (1985) Learning Logic. Technical Report-47, MIT Center for Computational Research in Economics and Management Science, Cambridge, MA, USA.
Pollack, J.B. (1987) Cascaded Back Propagation on Dynamic Connectionist Networks. In Proceedings of the Ninth Conference of the Cognitive Science Society. Seattle, 391–404
Pollack, J.B. (1988) Recursive Auto-Associative Memory: Devising Compositional Distributed Representations. Proceedings of the Tenth Conference of The Cognitive Science Society, Montreal, pp. 33–39.
Rosenblatt, F. (1959) Two theorems of statistical separability in the perceptron. In Mechanization of Thought Processes, Vol. 1. H.M.S.O., London.
Rosenblatt, F. (1962) Principles of Neurodynamics, Spartan, New York.
Rumelhart, W.E., Hinton, G.E. & Williams, R.J. (1986) Learning internal representations by error propagation. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1 (eds J.L.McClelland, D.E.Rumelhart & the PDP research Group). MIT Press, Cambridge.
Rumelhart, D.E. & McClelland, J.L. (1982) An interactive activation model of the effect of context in perception: Part 2 The contextual enhancement effect and some tests and extensions of the model. Psychology Review, 89, 60–94.
Rumelhart, D.E. & Zipser, D. (1986) Feature Discovery by Competitive Learning. In D.E.Rumelhart, J.L.McClelland & the PDP research Group, (eds.), Parallel Distributed Processing: Experiments in the Microstructure of Cognition, Vol. 1. MIT Press, Cambridge, USA.
Rumelhart, D.E. & McClelland, J.L. (1986) On Learning the Past Tenses of English Verbs. In Parallel Distributed Processing: Experiments in the Microstructure of Cognition, Vol. 2. J.L.McClelland, D.E.Rumelhart & the PDP research Group, (eds.), MIT Press, Cambridge, USA.
Sabbah, D. (1982) A Connectionist Approach to Visual Recognition. TR107: Computer Science Department, University of Rochester, USA.
Sampson, G. (1986) A stochastic approach to parsing. In Proceedings of the 11th International Conference on Computational Linguistics. pp. 151–155. Bonn.
Sejnowski, T.J. & Rosenberg, C.R. (1986) NETtalk: A parallel network that learns to read aloud. JHU/EECS-86/01: The Johns Hopkins University, Electrical Engineering and Computer Science Department.
Selman, B. (1985) Rule-Based Processing in a Connectionist System for Natural Language Understanding. CSRI-168, Toronto, Computer Systems Research Institute, University of Toronto, Canada.
Shastri, L. (1985) Evidential reasoning in semantic networks: A formal theory and its parallel implementation. TR166, Rochester: Computer Science Department, University of Rochester.
Smolensky, P. (1986) Information Processing in Dynamical Systems: Foundations of Harmony Theory. In Parallel Distributed Processing: Experiments in the Microstructure of Cognition, Vol. 1. D.E.Rumelhart, J.L.McClelland & the PDP research Group, (eds.), MIT Press, Cambridge, USA.
Smolensky, P. (1988) On the proper treatment of Connectionism. Behavioural and Brain Sciences, II, no. 1, 1–23.
Stornetta, W.S. & Huberman, B.A. (1987) An Improved three-layer back propagation algorithm. In Institute of Electrical and Electronics Engineers First International Conference on Neural Networks. San Diego, II-637-644.
Touretzky, D.S. & Hinton, G.E. (1985) Symbols among the neurons: details of a connectionist inference architecture. In Proceedings of the Ninth International Joint Conference on Artificial Intelligence. Los Angeles, CA, USA.
Willshaw, D.J. (1981) Holography, Associative Memory, and Inductive Generalization. In G.E.Hinton & J.A.Anderson, (eds.), Parallel models of associative memory. Lawrence Erlbaum Associates, Hillsdale, USA.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Pollack, J.B. Connectionism: past, present, and future. Artif Intell Rev 3, 3–20 (1989). https://doi.org/10.1007/BF00139193
Issue Date:
DOI: https://doi.org/10.1007/BF00139193