Noise enhancement of signal transduction by parallel arrays of nonlinear neurons with threshold and saturation

doi:10.1016/j.neucom.2006.12.014

Neurocomputing

Volume 71, Issues 1–3, December 2007, Pages 333-341

https://doi.org/10.1016/j.neucom.2006.12.014 Get rights and content

Abstract

A classic model neuron with threshold and saturation is used to form parallel uncoupled neuronal arrays in charge of the transduction of a periodic or aperiodic noisy input signal. The impact on the transduction efficacy of added noises is investigated. In isolated neurons, improvement by noise is possible only in the subthreshold and in the strongly saturating regimes of the neuronal response. In arrays, improvement by noise is always reinforced, and it becomes possible in all regimes of operation, i.e. in the threshold, in the saturation, and also in the intermediate curvilinear part of the neuronal response. All the configurations of improvement by noise apply equally to periodic and to aperiodic signals. These results extend the possible forms of stochastic resonance or improvement by noise accessible in neuronal systems for the processing of information.

Introduction

Neurons interconnected in networks are very efficient for signal and information processing, through detailed modalities and mechanisms which are still under intense investigation. Neurons are intrinsically nonlinear devices. It is now known that in nonlinear processes, the presence or even the injection of noise, can play a beneficial role for signal and information processing. This type of useful-noise phenomena have been widely investigated under the denomination of stochastic resonance [15], [1]. Many forms of stochastic resonance or improvement by noise have been reported in various nonlinear systems involved in diverse signal processing operations. Several forms of stochastic resonance have been reported in neural processes (see for instance [14], [12] for early experimental demonstrations, and [21] for a recent overview). At the level of the nonlinear neuron, many reported instances of stochastic resonance essentially rely on the threshold or excitable dynamics inherent to the neuron. In such situations, there is usually a small information-carrying signal, which is by itself too weak to elicit an efficient response from the threshold or excitable dynamics. The noise then cooperates constructively with the small signal, in such a way as to elicit a more efficient neuronal response.

Recently, a new mechanism of stochastic resonance has been exhibited when threshold or excitable nonlinearities are assembled into an uncoupled parallel array. This new form has been introduced under the name of suprathreshold stochastic resonance in [25], [26], because in the array, addition of noise can improve the transmission of an input signal with arbitrary amplitude, not necessarily a small subthreshold signal. A parallel array is a common architecture for neuron assemblies, especially in sensory systems in charge of the transduction of noisy signals from the environment. Stochastic resonance has been shown possible in neuronal parallel arrays, with various models for the threshold or excitable nonlinear dynamics of the neuron. In neuronal arrays, Collins et al. [11], Chialvo et al. [9], and Hoch et al. [18], [19] show stochastic resonance essentially with a subthreshold input signal, while Stocks [24], Stocks and Mannella [27], and Hoch et al. [17] show the novel form of suprathreshold stochastic resonance. In the neuronal arrays, suprathreshold stochastic resonance is shown in [24], [17] with simple threshold binary neurons, meanwhile Collins et al. [11], and Stocks and Mannella [27] for this investigate an excitable FitzHugh–Nagumo model in its subthreshold and suprathreshold regimes.

For isolated nonlinear systems, it has recently been shown that stochastic resonance can also operate in threshold-free nonlinearities with saturation, where the noise has the ability to reduce the distortion experienced by a signal because of the saturation [23], with an extension to arrays of threshold-free sensors with saturation given in [8]. Saturation is also a feature present in the neuronal response, and this effect of stochastic resonance at saturation has been shown to occur [22] in the transmission by a nonlinear neuron in its saturating region. More detailedly, Rousseau and Chapeau-Blondeau [22] demonstrate that in signal transmission by an isolated neuron, improvement by noise can take place both in the region of the threshold and in the region of the saturation; in between, when the neuron operates in the intermediate region avoiding both the threshold and the saturation, then improvement by noise does not take place. In the present paper, we shall consider the same type of neuron model with saturation as in [22]; we shall assemble these neurons into a parallel array, and investigate the impact of added noise in the array. We shall exhibit that different occurrences of stochastic resonance take place in the array. We shall show in the array that stochastic resonance is present in the threshold and in the saturation regimes of the neuronal response, just like in the case of an isolated neuron, but always with an increased efficacy brought in by the array. In addition, we shall show that in the array, stochastic resonance also takes place in the intermediate regime of operation that avoids both the threshold and the saturation of the neuron. Stochastic resonance does not arise in isolated neurons in this regime, but the property becomes possible in neuronal arrays through a truly specific array effect.

Section snippets

The model of neuronal array

We consider the neuron model of [22]. The input signal to the neuron, at time t, is taken as the total somatic current $I (t)$ . This input current $I (t)$ may result from presynaptic neuronal activities, or also from an external stimulus of the environment for sensory neurons, a situation to which stochastic resonance effects are specially relevant. The output response of the neuron is taken as the short-term firing rate $f (t)$ at which action potentials are emitted in response to $I (t)$ . A classic

Assessing nonlinear transmission by the array

To demonstrate a neuronal transmission aided by noise, we consider that the input current $I (t)$ to the array is formed as $I (t) = s (t) + ξ (t) .$ In Eq. (4), $s (t)$ is our information-carrying signal, which will be successively considered to be a periodic and an aperiodic component. $s (t)$ conveys an image of the information coming from presynaptic neurons or from the external world for sensory cells. Also in Eq. (4), $ξ (t)$ is a white noise, independent of $s (t)$ and of the $η_{i} (t)$ , with probability density

Array transmission aided by noise

Direct numerical evaluations of $R_{out}$ and $C_{sy}$ can be realized through numerical integration of the integrals of Eqs. (9), (10), (11), (12). Alternatively, to push further the analytical treatment, it is possible to consider the following situation. The Lapicque function $g (.)$ of Eq. (1) can be approximated for $I (t)$ sufficiently above the threshold $I_{th}$ by using $\ln (1 - I_{th} / I) \approx - I_{th} / I$ , yielding the approximation $f (t) = g [I (t)] = \{\begin{matrix} 0 & for I (t) ⩽ I_{th}, \\ \frac{1 / T_{r}}{1 + (τ_{m} / T_{r}) (I_{th} / I (t))} & for I (t) > I_{th}, \end{matrix}$ which is also depicted in Fig. 1

Discussion

The results presented here can be viewed as extensions concerning the various forms of stochastic resonance or improvement by noise in signal transduction by neurons. In this context, stochastic resonance has been widely studied for noise-enhanced signal transmission by neurons operating in the region of the threshold of activity. Here, by contrast, we have also investigated signal transmission away of the threshold, in the curvilinear part of the neuronal response, and beyond in the saturation

Solenna Blanchard received a M.Sc. degree in Signal Processing in 2004 and a M.Sc. degree in Electronics in 2005, both from the University of Rennes, France. She is currently a Ph.D. student in the field of nonlinear signal processing and stochastic resonance at the University of Angers, France.

References (27)

F. Chapeau-Blondeau et al.
Noise-aided SNR amplification by parallel arrays of sensors with saturation
Phys. Lett. A
(2006)
T. Hoch et al.
Adaptation using local information for maximizing the global cost
Neurocomputing
(2003)
T. Hoch et al.
The effect of correlations in the background activity on the information transmission properties of neural populations
Neurocomputing
(2005)
F. Moss et al.
Stochastic resonance and sensory information processing: a tutorial and a review of application
Clin. Neurophysiol.
(2004)
B. Andò et al.
Stochastic Resonance: Theory and Applications
(2000)
N.M. Blachman
The output signal-to-noise ratio of a power-law device
J. Appl. Phys.
(1953)
J.M. Casado, J. Gómez Ordóñez, M. Morillo, Stochastic resonance of collective variables in finite sets of interacting...
F. Chapeau-Blondeau et al.
Dynamic properties of a biologically motivated neural network model
Int. J. Neural Syst.
(1992)
F. Chapeau-Blondeau et al.
Theory of stochastic resonance in signal transmission by static nonlinear systems
Phys. Rev. E
(1997)
F. Chapeau-Blondeau, D. Rousseau, Enhancement by noise in parallel arrays of sensors with power-law characteristics,...

F. Chapeau-Blondeau et al.

Nonlinear SNR amplification of a harmonic signal in noise

Electron. Lett.

(2005)

D.R. Chialvo et al.

Stochastic resonance in models of neuronal ensembles

Phys. Rev. E

(1997)

J.J. Collins et al.

Aperiodic stochastic resonance in excitable systems

Phys. Rev. E

(1995)

Cited by (21)

Enhancing threshold neural network via suprathreshold stochastic resonance for pattern classification
2021, Physics Letters, Section A: General, Atomic and Solid State Physics
Hard-threshold nonlinearities are of significant interest for neural-network information processing due to their simplicity and low-cost implementation. They however lack an important differentiability property. Here, hard-threshold nonlinearities receiving assistance from added noise are pooled into a large-scale summing array to approximate a neuron with a noise-smoothed activation function. Differentiability that facilitates gradient-based learning is restored for such neurons, which are assembled into a feed-forward neural network. The added noise components used to smooth the hard-threshold responses have adjustable parameters that are adaptively optimized during the learning process. The converged non-zero optimal noise levels establish a beneficial role for added noise in operation of the threshold neural network. In the retrieval phase the threshold neural network operating with non-zero optimal added noise, is tested for data classification and for handwritten digit recognition, which achieves state-of-the-art performance of existing backpropagation-trained analog neural networks, while requiring only simpler two-state binary neurons.
Positive role of fractional Gaussian noise in FitzHugh–Nagumo neuron model
2021, Chaos, Solitons and Fractals
Citation Excerpt :
The research of SR is helpful for the neuroscientists to understand how the brain encodes and transmits information from the sensory stimuli to the spike activity of the neurons. It is always measured by signal-to-noise ratio (SNR) or mutual information [9,10]. SR often referred to as positive role of noise.
Determining the complex mechanisms of process information in the neural activity is found to be especially challenging. The noise with a 1/f power spectrum has been observed in nervous system, but its functional significance in the neuron activity remains unclear. Persistent effort has been made to determine the factors for efficient processing of information and for promoting the mutual information between stimulus and spike train output. Establishing the Fitzhugh–Nagumo (FHN) model coupling fractional Gaussian noise (fGn) as a special form of stochastic differential equation, our study is to certify the stochastic resonance (SR) effect in the process of neuron activity. We proved that the nonmonotonic SR effect about FHN neuron model driven by fGn occurred under the sufficient conditions based on the principle of forbidden interval. The simulated results show that appropriate intensity of fGn can enhance the increase of the mutual information. Compared with the Hurst parameters of fGn, the increase is more dependent on the noise intensity of fGn.
Coupled neurons with multi-objective optimization benefit incipient fault identification of machinery
2021, Chaos, Solitons and Fractals
Organisms can sense subtle changes in the environment around them such as temperature, vibration and magnetic field. That is because biological neural network interconnected millions of neurons by synapses is able to utilize noise to amplify such subtle changes, and then encode and transmit them to make the corresponding biological responses. Such favorable use of noise can be improved by coupling two different neurons like synapses in organisms to enhance weak useful signature identification. Inspired by above mechanism, we investigate the benefits of noise in the coupled neurons to weak useful signature identification and then propose an adaptive coupled neurons-based method with multi-objective optimization to enhance incipient fault signature identification of machinery for overcoming the drawback that blind noise suppression and cancellation using unwanted noise techniques not only are prone to remove weak useful signature closely related with health states of machinery but also are impossible to cancel strong background noise. In the proposed method, both signal-to-noise ratio (SNR) and residence-time distribution ratio are seen as the multi-objective function to optimize the adjusting parameters of the coupled neurons and rescaling factor simultaneously by using genetic algorithms. Finally, two rolling element bearing experiments including a double row bearing run-to-failure experiment and a high-speed train bearing experiment were performed to demonstrate the feasibility and effectiveness of the proposed method in mechanical incipient fault diagnosis. The experimental results show that the proposed method not only enhances weak fault signature identification of machinery by coupling two different neurons but also is superior to the filter-based methods.
Vibrational resonance without tuning in a neuronal parallel array
2019, Physica A: Statistical Mechanics and its Applications
Citation Excerpt :
Parallel structure makes the propagation of the signal more robust. SR phenomena in neuronal parallel arrays have been investigated with various models [21–24]. Additionally, aperiodic stochastic resonance (ASR), which refers to the aperiodic input signals can characterize the SR-type behavior in excitable system, is also widely studied [5,25–27].
This paper investigates the propagation of the weak aperiodic signal in a parallel array of FitzHugh–Nagumo (FHN) neurons with heterogenous aperiodic high-frequency (HAHF) disturbances. Different from the traditional vibrational resonance, where the optimal amplitudes of the HF driving signal should be tuned for individual elements, the ability of the proposed averaging parallel array network for weak signal detection can be optimized at a fixed amplitude of the HAHF disturbance, regardless of the nature of the input signal. Local connections in the parallel array network are found to be important for the propagation of weak signal in parallel array. Besides, the characteristics of high-frequency signal such as heterogeneity and frequency, can also modulate the propagation of aperiodic signal in parallel array.
Optimising threshold levels for information transmission in binary threshold networks: Independent multiplicative noise on each threshold
2015, Physica A: Statistical Mechanics and its Applications
The problem of optimising the threshold levels in multilevel threshold system subject to multiplicative Gaussian and uniform noise is considered. Similar to previous results for additive noise, we find a bifurcation phenomenon in the optimal threshold values, as the noise intensity changes. This occurs when the number of threshold units is greater than one. We also study the optimal thresholds for combined additive and multiplicative Gaussian noise, and find that all threshold levels need to be identical to optimise the system when the additive noise intensity is a constant. However, this identical value is not equal to the signal mean, unlike the case of additive noise. When the multiplicative noise intensity is instead held constant, the optimal threshold levels are not all identical for small additive noise intensity but are all equal to zero for large additive noise intensity. The model and our results are potentially relevant for sensor network design and understanding neurobiological sensory neurons such as in the peripheral auditory system.
Spurious correlation as an approximation of the mutual information between redundant outputs and an unknown input
2014, Communications in Nonlinear Science and Numerical Simulation
Citation Excerpt :
This setting takes a very important role in engineering applications [8,7,10,19], where a single unknown signal is often measured by multiple sensing elements. The same setting assumes great importance in biology as well [16,20], because information is often coded and transmitted using populations of neurons [21–23]. In fact, it was shown that information transmission through multiple low capacity pathways turns out to be more efficient, in terms of metabolic cost, than a single high bit-rate communication channel.
Stochastic resonance (SR) is a counterintuitive phenomenon, observed in a wide variety of nonlinear systems, for which the addition of noise of opportune magnitude can improve signal detection. Tuning the noise for maximizing the SR effect is important both for artificial and biological systems. In the case of artificial systems, full exploitation of the SR effect opens the possibility of measuring otherwise unmeasurable signals. In biology, identification of possible SR maximization mechanisms is of great interest for explaining the low-energy high-sensitivity perception capabilities often observed in animals. SR maximization approaches presented in literature use knowledge on the input signal (or stimulus, in the case of living beings), and maximize the mutual information between the input and the output signal. The input signal, however, is unknown in many practical settings. To cope with this problem, this paper introduces an approximation of the input–output mutual information based on the spurious correlation among a set of redundant units. A proof of the approximation, as well as numerical examples of its application are given.

View all citing articles on Scopus

David Rousseau was born in 1973 in France. He received the Master degree in acoustics and signal processing from the Institut de Recherche Coordination Acoustique et Musique (IRCAM), Paris, France in 1996. He received, in 2004, the Ph.D. degree in nonlinear signal processing and stochastic resonance at the Laboratoire d’Ingénierie des Systèmes Automatisés (LISA), University of Angers where he is currently a Maître de Conférences of physics and information sciences.

François Chapeau-Blondeau was born in France in 1959. He received the Engineer Diploma from ESEO, Angers, France, in 1982, the Ph.D. degree in electrical engineering from University Paris 6, Paris, France, in 1987, and the Habilitation degree from the University of Angers, France, in 1994. In 1988, he was a research associate in the Department of Biophysics at the Mayo Clinic, Rochester, Minnesota, USA, working on biomedical ultrasonics. Since 1990, he has been with the University of Angers, France, where he is currently a professor of electronic and information sciences. His research interests are nonlinear systems and signal processing, and the interactions between physics/biophysics and information processing, including neural information processing.

View full text

Noise enhancement of signal transduction by parallel arrays of nonlinear neurons with threshold and saturation

Abstract

Introduction

Section snippets

The model of neuronal array

Assessing nonlinear transmission by the array

Array transmission aided by noise

Discussion

Phys. Lett. A

Neurocomputing

Neurocomputing

Clin. Neurophysiol.

Stochastic Resonance: Theory and Applications

The output signal-to-noise ratio of a power-law device

J. Appl. Phys.

Dynamic properties of a biologically motivated neural network model

Int. J. Neural Syst.

Theory of stochastic resonance in signal transmission by static nonlinear systems

Phys. Rev. E

Nonlinear SNR amplification of a harmonic signal in noise

Electron. Lett.

Stochastic resonance in models of neuronal ensembles

Phys. Rev. E

Aperiodic stochastic resonance in excitable systems

Phys. Rev. E