Adaptive synaptogenesis constructs networks that maintain information and reduce statistical dependence

Abstract

This report demonstrates the effectiveness of two processes in constructing simple feedforward networks which perform good transformations on their inputs. Good transformations are characterized by the minimization of two information measures: the information loss incurred with the transformation and the statistical dependency of the output. The two processes build appropriate synaptic connections in initially unconnected networks. The first process, synaptogenesis, creates new synaptic connections; the second process, associative synaptic modification, adjusts the connection strength of existing synapses. Synaptogenesis produces additional innervation for each output neuron until each output neuron achieves a firing rate of approximately 0.50. Associative modification of existing synaptic connections lends robustness to network construction by adjusting suboptimal choices of initial synaptic weights. Networks constructed using synaptogenesis and synaptic modification successfully preserve the information content of a variety of inputs. By recording a high-dimensional input into an output of much smaller dimension, these networks drastically reduce the statistical dependence of neuronal representations. Networks constructed with synaptogenesis and associative modification perform good transformations over a wide range of neuron firing thresholds.