Implementation of multilayer perceptron networks by populations of spiking neurons using rate coding

doi:10.1016/S0925-2312(02)00381-8

Neurocomputing

Volumes 44–46, June 2002, Pages 353-358

https://doi.org/10.1016/S0925-2312(02)00381-8 Get rights and content

Abstract

In this paper we consider the biological plausibility of perceptron networks. We identify problems that arise if one tries to model form processing in visual cortex with a perceptron network. We conclude that a perceptron's activity corresponds to a population firing rate and that serious rate problems can be avoided if the squashing function maps zero input on zero output. Typically, such squashing functions are anti-symmetric. We present a circuit with a perceptron-like behavior, which provides an elegant interpretation for the negative perceptron activities that these functions entail.

Section snippets

Motivation

Multilayer feedforward perceptron networks are widely used in modeling higher cognitive functions. One particular example is form processing in visual cortex. Visual cortex processes familiar objects so fast, that this is believed to be a feedforward process. Therefore, visual cortex is able to perform a non-trivial computation, like object recognition or classification, using cortical networks that are massively parallel and that use the same basic element, the neuron, over and over again. The

Perceptron activity as a population rate

Originally, the perceptron was introduced as a highly abstracted version of a real neuron. Its state o is described in terms of its input signals i_j, input weights w_j and given by $o=f(Σ_{j} w_{j} i_{j} −θ).$ Here f is a non-descending squashing function and θ a threshold. To identify the perceptron activity with that of a neuron, a time-averaged rate hypothesis could be considered: the perceptron activity represents the firing rate of a real neuron averaged over a short window in time. There are a number of

Biological considerations

A typical characteristic of cortical networks is that the firing rates of the neurons are low. This precludes the use of squashing functions like $f(x)= 1 1+ exp (−βx) .$ This may come as a surprise, because Eq. (3) is perhaps the most widely used squashing function in cognitive modeling, but the reason is simple: f(0)=0.5. If the perceptron's activity corresponds to a firing rate, then this implies that a population that receives no input will fire at half of its maximum firing rate, which is well

The interpretation of the squashing function

The first squashing function that comes to mind, which maps 0 on zero output is $f(x)= 2 1+ e^{−βx} −1.$ Indeed, with this squashing function the rate problems are readily avoided. It leaves the question, however, of how to interpret the negative branch of the squashing function. We propose the following circuit to implement this squashing function. The idea is that population P codes for the positive branch of the squashing function and N for the negative branch. The cross-inhibition ensures that only

Discussion

Clearly, there are better choices for spike response functions than the ones we used in the last section. They were picked to demonstrate the perceptron-like behavior of the circuit. A more realistic choice leads to a different kind of perceptron with interesting effects, in particular to the fact that this perceptron loses its discriminatory power for large inputs, even if there are large differences between the inputs [3]. Dale's law is respected in a natural way. The circuit is symmetric and

References (4)

M. de Kamps et al.
From artificial neural networks to spiking neuron populations and back again
Neural Networks
(2001)
W. Gerstner
Time structure in the activity in neural network models
Phys. Rev. E
(1995)

There are more references available in the full text version of this article.

Cited by (2)

Computing and recomputing discourse models: An ERP study
2008, Journal of Memory and Language
While syntactic reanalysis has been extensively investigated in psycholinguistics, comparatively little is known about reanalysis in the semantic domain. We used event-related brain potentials (ERPs) to keep track of semantic processes involved in understanding short narratives such as ‘The girl was writing a letter when her friend spilled coffee on the paper’. We hypothesize that these sentences are interpreted in two steps: (1) when the progressive clause is processed, a discourse model is computed in which the goal state (a complete letter) is predicted to hold; (2) when the subordinate clause is processed, the initial representation is recomputed to the effect that, in the final discourse structure, the goal state is not satisfied. Critical sentences evoked larger sustained anterior negativities (SANs) compared to controls, starting around 400 ms following the onset of the sentence-final word, and lasting for about 400 ms. The amplitude of the SAN was correlated with the frequency with which participants, in an offline probe-selection task, responded that the goal state was not attained. Our results raise the possibility that the brain supports some form of non-monotonic recomputation to integrate information which invalidates previously held assumptions.
Supervised Learning Strategy for Spiking Neurons Based on Their Segmental Running Characteristics
2023, Neural Processing Letters

View full text