Pattern analysis with spiking neurons using delay coding
Introduction
Radial basis functions (RBFs), which have turned out to be among the most powerful artificial neural network types are also receiving increased attention within biological neural networks. The question if biological neurons can realize one of the main advantages of RBF neurons, namely their ability to discover clusters in the input space is not yet resolved. We present a learning algorithm for spiking neurons realizing RBFs which is not based on rate coding but on the timing of single spikes. There exists substantial evidence that this type of temporal coding is important for biological systems, especially for fast neural information processing [6]. In this context, Hopfield presented a model for computing RBFs with spiking neurons [2]. The basic idea is that an “RBF neuron” encodes a particular input spike pattern in the delays available across its synapses. If an input pattern is close to the encoded spike pattern of an RBF neuron, the delays even out the differences of the firing times of the input neurons such that the RBF neuron fires. We extend this approach by considering not only the firing/non-firing of a neuron but also its firing time. We show that on the basis of this idea a network of such RBF neurons can be constructed and trained to perform several pattern analyzing tasks.
Section snippets
Computing and learning radial basis functions
We consider a network of spiking neurons of the leaky integrate-and-fire type with input neurons u1,…,um and RBF neurons v1,…vn. In the simplest case each input neuron ui forms one synaptic connection with each RBF neuron vj with weight wji and delay dji, where the delay is given by the difference between the presynaptic firing time and the time the synapse between the two neurons is activated. We assume that the postsynaptic delay is always constant and neglect it here for the sake of
Finding clusters in subspaces
If one interprets the input coordinates as a representation of certain features, then the clusters represent typical constellations of frequently occurring combinations of input values. A cluster may then be formed out of some features, whereas other features are irrelevant. In our context, this means that only certain input neurons describe a cluster whereas the remaining input neurons simply produce noise. The task of the RBF neuron is to extract the “relevant” inputs for every cluster.
More
Temporal sequence recognition
Since the input patterns are temporally encoded in our approach, it is quite natural to use it for the recognition of regularities in temporal sequences of input stimuli. This is of particular importance in biological systems (e.g. in audition and vision) but also for related tasks in hardware implementations. We will show that it is possible to recognize sequences which are distorted in time and form.
The detection of spatio-temporal input firing patterns corresponds to the clustering problem
Conclusions
We extended Hopfield's [2] idea by considering not only the firing/non-firing of a neuron but also its firing time. We have seen that on the basis of this idea a network of such RBF neurons can be constructed and trained to divide the input space into several clusters, where the selection of the proper delays uses exclusively the difference between pre- and post-synaptic firing time. Some of our main results are: (a) The RBF neurons converged very reliably to the centers of the clusters even in
Thomas Natschläger is a Ph.D. student at the University of Technology Graz, Austria. He was born in Wels, Austria, on 1 February 1970. He studied computer science at the University of Technology in Graz, Austria where he received his diploma in 1996. Since then he is a research assistant at the Institute for Theoretical Computer Science at the same university associated with the FWF (Austrian Science Fund) project “Computing and Learning in Networks of Spiking Neurons”.
References (6)
- et al.
A neuronal learning rule for sub-millisecond temporal coding
Nature
(1996) Pattern recognition computation using action potential timing for stimulus representation
Nature
(1995)- et al.
Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs
Science
(1997)
Cited by (0)
Thomas Natschläger is a Ph.D. student at the University of Technology Graz, Austria. He was born in Wels, Austria, on 1 February 1970. He studied computer science at the University of Technology in Graz, Austria where he received his diploma in 1996. Since then he is a research assistant at the Institute for Theoretical Computer Science at the same university associated with the FWF (Austrian Science Fund) project “Computing and Learning in Networks of Spiking Neurons”.
Berthold Ruf was born in Düren, Germany, on 14 January 1967. He studied computer science at the University of Technology in Aachen, Germany where he received his diploma in 1993. In 1993/94 he was a visiting scientist at the school of computer science at the Carleton University, Ottawa, Canada, which was made possible by a grant of the German Academic Exchange Service (DAAD). Since 1994 he is with the Institute for Theoretical Computer Science at the University of Technology Graz, Austria, where he received his Ph.D. in 1998.
- 1
T.N. acknowledges support by the “Fonds zur Förderung der wissenschaftlichen Forschung (FWF), Austrian Science Fund”, project number P12153.