Why Distributed Representation is Inherently Non-Symbolic

Abstract

There are many conflicting views concerning the nature of distributed representation, its compatibility or otherwise with symbolic representation, and its importance in characterizing the nature of connectionist models and their relationship to more traditional symbolic approaches to understanding cognition. Many have simply assumed that distribution is merely an implementation issue, and that symbolic mechanisms can be designed to take advantage of the virtues of distribution if so desired. Others, meanwhile, see the use of distributed representation as marking a fundamental difference between the two approaches. One reason for this diversity of opinion is the fact that the relevant notions — especially that of distribution — are rarely adequately characterized before addressing the issues. At this level of generality, an adequate characterization is one that is sufficiently abstract to subsume most paradigm cases of representation of a given type, yet also sufficiently precise to give real theoretical bite when addressing questions such as those raised above. This paper advances a definition of distributed representation and shows that, understood this way, distribution is in fact incompatible with the core notion of symbolic representation found in the cognitive science literature. For this reason, genuinely distributed connectionist models cannot be, or implement, physical symbol systems (Newell & Simon 1976) or “classical” symbolic models (Fodor and Pylyshyn 1988). Thus, I am endorsing the view that distributed connectionist models do indeed present a radical new approach to modeling cognitive processes.