Abstract
Fodor's and Pylyshyn's stand on systematicity in thought and language has been debated and criticized. Van Gelder and Niklasson, among others, have argued that Fodor and Pylyshyn offer no precise definition of systematicity. However, our concern here is with a learning based formulation of that concept. In particular, Hadley has proposed that a network exhibits strong semantic systematicity when, as a result of training, it can assign appropriate meaning representations to novel sentences (both simple and embedded) which contain words in syntactic positions they did not occupy during training. The experience of researchers indicates that strong systematicity in any form is difficult to achieve in connectionist systems.
Herein we describe a network which displays strong semantic systematicity in response to Hebbian, connectionist training. During training, two-thirds of all nouns are presented only in a single syntactic position (either as grammatical subject or object). Yet, during testing, the network correctly interprets thousands of sentences containing those nouns in novel positions. In addition, the network generalizes to novel levels of embedding. Successful training requires a, corpus of about 1000 sentences, and network training is quite rapid. The architecture and learning algorithms are purely connectionist, but ‘classical’ insights are discernible in one respect, viz, that complex semantic representations spatially contain their semantic constituents. However, in other important respects, the architecture is distinctly non-classical.
Similar content being viewed by others
References
Anderson, J.R. (1977), ‘Induction of Augmented Transition Networks’, Cognitive Science1, pp. 125–157.
Barnden, J.A. and Srinivas, K. (1991), ‘Encoding Techniques for Complex Information Structures in Connectionist Systems’, Connection Science3, pp. 263–309.
Barnden, J.A. (1992 ), ‘Connectionism, Structure-Sensitivity, and Systematicity: Refining the Task Requirements’, Technical Report MCCS-92-227, Computing Research Laboratory, Box 30001, New Mexico State University, Las Cruces, NM, 88003.
Chalmers, D. (1990 ), ‘Why Fodor and Pylyshyn WereWrong: the Simplest Refutation,’ Proceedings of the Twelfth Annual Conference of the Cognitive Science Society, Cambridge, Mass. pp. 340–347.
Christiansen, M.H. and Chater, N. (1994), ‘Generalization and Connectionist Language Learning’, Mind and Language9, pp. 273–287.
Cottrell, G. (1985), ‘Connectionist Parsing’, Proceedings of the Seventh Annual Conference of the Cognitive Science, Society, Irvine, CA.
Elman, J.L. (1990), ‘Finding Structure in Time’, Cognitive Science14, pp. 179–212.
Feldman, J.A. and Ballard, D.H. (1982), ‘Connectionist Models and Their Properties’, Cognitive Science6, pp. 205–254.
Fodor, J.A. and McLaughlin, B.P. (1990), ‘Connectionism and the Problem of Systematicity: Why Smolensky's Solution Doesn't Work’, Cognition35, pp. 183–204.
Fodor, J.A. and Pylyshyn, Z.W. (1988 ), ‘Connectionism and Cognitive Architecture: a Critical Analysis, Cognition28, pp. 3–71.
Grossberg, S. (1976 ), ‘Adaptive Pattern Classification and Universal Recoding: Part I. Parallel Development and Coding of Neural Feature Detectors’, Biological Cybernetics23, pp. 121–134.
Hadley, R.F. (1989), ‘A Default-Based Theory of Procedural Semantics,’ Cognitive Science13, pp. 107–137.
Hadley, R.F. (1992), ‘Compositionality and Systematicity in Connectionist Language Learning’, Proceedings of the Fourteenth Annual Conference of the Cognitive Science Society, Bloomington, Indiana, pp. 659–664.
Hadley, R.F. (1993), ‘Connectionism, Explicit Rules, and Symbolic Manipulation’, Minds and Machines3, pp. 183–200.
Hadley, R.F. (1994a), ‘Systematicity in Connectionist Language Learning’, Mind and Language9, pp. 247–272.
Hadley, R.F. (1994b), ‘Systematicity Revisited: Reply to Christiansen and Chater and Niklasson and van Gelder’, Mind and Language9, pp. 431–444.
Hadley, R.F. (1995a), ‘The Explicit-Implicit Distinction’, Minds and Machines5, pp. 219–242.
Hadley, R.F. (1995b), ‘Systematicity and Nomic Necessity’, Technical Report CSS-IS TR 95-08, Centre for Systems Science, Simon Fraser University, Burnaby, B.C. V5A 1S6.
Hebb, D.O. (1949), The Organization of Behaviour, New York: Wiley.
Ingram, D. (1985), ‘The Psychological Reality of Children's Grammars and Its Relation to Grammatical Theory’, Lingua66, pp. 79–103.
Ingram, D. (1989), First Language Acquisition, Cambridge, UK: Cambridge University Press.
Lange, T.E. and Dyer, M.G. (1989), ‘High-Level Inferencing in a Connectionist Network’, Connection Science1, pp. 181–217.
McClelland, J.L. and Kawamoto, A.H. (1986), ‘Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences’, in D.E. Rumelhart, J.L McClelland and the PDP Research Group, eds., Parallel Distributed Processing; Explorations in the Microstructure of Cognition, Volume2, Cambridge, MA: MIT Press.
McLaughlin, B.P. (1993), ‘The Connectionism /Classicism Battle to Win Souls, Philosophical Studies71, pp. 163–190.
Moeser, S.D. and Bregman, A.S. (1973), ‘Imagery and Language Acquisition’, Journal of Verbal Learning and Verbal Behavior12, pp. 91–98.
Niklasson, L.F. and van Gelder, T. (1994 ), ‘On Being Systematically Connectionist’, Mind and Language9, pp. 288–302.
Phillips, S. (1994), ‘Strong Systematicity within Connectionism: the ‘Tensor-Recurrent Network’, Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society, Atlanta, GA, pp. 723–727.
Pinker, S. (1984 ), Language Learnability and Language Development, Cambridge, MA: Harvard University Press.
Pinker, S. (1989 ), Learnability and Cognition: the Acquisition of Argument Structure, Cambridge, MA: MIT Press.
Pinker, S., Lebeaux, D.S. and Frost, L.S. (1987), ‘Productivity and Constraints in the Acquisition of the Passive’, Cognition28, pp. 73–193.
Pollack, J.B. (1990 ), ‘Recursive Distributed Representations’, Artificial Intelligence46, pp. 77–105.
Rumelhart, D.E. and McClelland, J.L. (1986), ‘PDP Models and General Issues in Cognitive Science’, in D.E. Rumelhart, J.L McClelland and the PDP Research Group, eds., Parallel Distributed Processing; Explorations in the Microstructure of Cognition, Volume2,Cambridge, MA: MIT Press.
Schubert, L. (1976), Extending the Expressive Power of Semantic Networks’, Artificial Intelligence7, pp. 163–198.
Shastri, L. and Ajjanagadde, V. (1993), ‘From Simple Associations to Systematic Reasoning: A Connectionist Representation of Rules, Variables and Dynamic Bindings Using Temporal Synchrony’, Behavioral and Brain Sciences16, pp. 417–451.
Smolensky, P. (1988), ‘On the Proper Treatment of Connectionism’, Behavioral and Brain Sciences11, pp. 1–23.
Smolensky, P. (1990), ‘Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems’, Artificial Intelligence46, pp. 159–216.
Smolensky, P., Legendre, G. and Miyata, Y. (1992), ‘Principles for an Integrated Connectionist/ Symbolic Theory of Higher Cognition’, Tech. Report CU-CS-600-92, Computer Science Department, University of Colorado at Boulder.
Smolensky, P. (1994), ‘Constituent Structure and Explanation in an Integrated Connectionist/ Symbolic Cognitive Architecture’, The Philosophy of Psychology: Debates on Psychological Explanation, C. Macdonald and G. Macdonald, eds., Oxford: Basil Blackwell.
Stevenson, S. (1994 ), ‘A Unified Model of Preference and Recovery Mechanisms in Human Parsing’, Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society, Atlanta, GA, pp. 824–829.
St. John, M.F. and McClelland, J.L. (1990), ‘Learning and Applying Contextual Constraints in Sentence Comprehension’, Artificial Intelligence46, pp. 217–257.
Sun, R. (1993), ‘Beyond Associative Memories: Logics and Variables in Connectionist Models’, Information Sciences70, pp. 49–73.
van Gelder, T. and Niklasson, L.F. (1994), ‘Classicalism and Cognitive Architecture’ Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society, Atlanta, Georgia, pp. 905–909.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hadley, R.F., Hayward, M.B. Strong Semantic Systematicity from Hebbian Connectionist Learning. Minds and Machines 7, 1–37 (1997). https://doi.org/10.1023/A:1008252408222
Issue Date:
DOI: https://doi.org/10.1023/A:1008252408222