Connectionism and Language Processing

Abstract

A survey of the field of connectionist work on language reveals a somewhat daunting variety of architectures, formalisms and models. Partly this diversity just reflects the breadth of the domain: language processing embraces everything from phoneme recognition to such areas as parsing, sentence disambiguation, story understanding, and speech generation. Human language processing is a vast complex of distinguishable capacities, and connectionism has been busily chipping away at any aspect that seems to lend itself to neural processing. However, the diversity of connectionist efforts also reflects some real theoretical differences over what kinds of mechanisms one needs, in principle, to be able to generate plausible models and hence adequate explanations. It is an increasingly common observation that, at one end of the connectionist spectrum, there are those who feel that no serious account of language processing can hope to avoid relying on explicit rules, some measure of serial processing, complexly structured representations, variable binding and so forth, and regard neural networks as a useful new way to implement such mechanisms; while at the other end there are those who regard connectionist methods as an excellent excuse to avoid all such baroque entanglements, and hope that self-generated, dynamic, distributed, gestalt-style representations will be the only necessary intermediaries between the ear and the vocal chords. People often think of this as a division between the conservatives — those still wedded to old-fashioned devices handed down from previous digital, serial, symbolic paradigms — and the radicals, those prepared to embrace fundamentally new approaches. (As in politics, of course, the conservatives see things slightly differently: they tend to regard themselves as the responsible realists, while the radicals are crazed utopian idealists.)