Neural net connection estimates applied for feature selection & improved linear classifier design

Abstract

Estimates of connection strengths are obtained to derive the weights used in building a linear classifier of complex binary patterns. Use of bootstrap resampling methods permits small, large and highly disparate-in-size training sets to be utilized with equal ease. Using ideas suggested by a method proposed for learning in neural net systems [Hopfield 1982] [Cruz-Young et al 1986], it is now possible to obtain connection values without prohibitive or arbitrarily terminated computations. The weight values so derived are identified as equivalent with those at the middle layer of three-layer neural net models. The model serving as the springboard for this study was developed by Kanerva [1986–87], and functions as a distributed sparse memory or DSM. In the various implementations of connection-based memories, both linear and nonlinear relationships within patterns play a role in the classification process. Experimental data used in this research were derived from psychological profiles which result from the coding of responses elicited by question-like items. Traditionally, such items are usually designed and chosen to be linear predictors of class membership or performance. The phase of the study reported here addresses the problems and successes of initial efforts towards a practical application of neural network computational concepts. The main focus is on those approaches which have been found to be particularly effective in obtaining adequate estimates of effective linear functions for classification of binary patterns with a number of hits ranging from 64 to 256.