Abstract
We investigated the normalized autocovariance (correlation coefficient) function of the output of an erf( ) function nonlinearity subject to non-zero mean Gaussian noise input. When the sigmoid is wide compared to the input, or the input mean is close to the midpoint of the sigmoid, the output correlation coefficient function is very close to the input correlation coefficient function. When the noise mean and variance are such that there is a significant probability of operating in the saturation region and the sigmoid is not too flat, the correlation coefficient of the output function is less than that of the input. This difference is much greater when the correlation coefficient is negative than when it is positive. The sigmoid partially rectifies the correlation coefficient function.
The analysis does not depend on the spectral properties of the input noise. All that is required is that the input at times t and (t + τ) be jointly gaussian with the same mean and autocovariance. The analysis therefore applies equally well to the case of two identical sigmoids with jointly gaussian inputs. This correlational rectification could help explain the parameter sensitivity of “neural network” models. If biological neurons share this property it could explain why few negative correlations between spike trains-have been observed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abeles M (1991). Corticonics: neural circuits of the cerebral cortex. Cambridge University Press, Cambridge, UK
Aertsen AMHJ, Gerstein GL (1985) Evaluation of neuronal connectivity: sensitivity of cross-correlation. Brain Res. 340:341–354
Baum RF (1957) The correlation function of smoothly limited gaussian noise. IRE Trans Inform Theory 3:193–197
Bedenbaugh P (1993) Plasticity in the rat somatosensory cortex induced by local microstimulation and theoretical investigations of information flow through neurons. PhD thesis, University of Pennsylvania
Freeman WJ (1967) Analysis of function of cerebral cortex by use of control systems theory. Logistics Rev. 3(12):5–40
Freeman WJ (1968) Analog simulation of prepyriform cortex in the cat. Math. Biosci. 2:181–190
Gray CM, König P, Engel AK, Singer W (1989) Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature 338:334–337
Grossberg S (1973) Contour enhancement, short term memory, and constancies in reverberating neural networks. Stud Appl Math 52(3):213–257
Jenkins GM, Watts DG (1968) Spectral analysis and its applications. Holden-Day, San Francisco
Johannesma PIM, Boogard HFP van den (1985) Stochastic formulation of neural interaction. Acta Appl Math 4:201–224
Kernell D (1965a) High-frequency repetitive firing of cat lumbosacral motoneurones stimulated by long-lasting injected currents. Acta Physiol Scand 65:74–86
Kernell D (1965b) The limits of firing frequency in cat lumbosacral motoneurones possessing different time course of afterhyperpolarization. Acta Physiol Scand 65:87–100
Kohonen T (1987) Self-organization and associative memory, 2nd edn. Springer, New York Berlin Heidelberg
Leipnik R (1958) The effect of instantaneous nonlinear devices on crosscorrelation. IRE Trans Inform Theory 4:73–76
Little WA (1974) The existence of persistent states in the brain. Math Biosci. 19:101–120
Marmarelis PZ, Marmarelis VZ (1978) Analysis of physiological systems, Wiley, New York
Papoulis A (1984) Probability, random variables, and stochastic processes, 2nd edn. McGraw-Hill, New York
Piessens R, Doncker-Kapenga E de, Überhuber CW, Kahaner DK (1983) Quadpack: a subroutine package for automatic integration. Springer Series in Computational Mathematics, vol 1 Springer, Berlin Heidelberg New York
Price R (1958) A useful theorem for nonlinear devices having Gaussian inputs. IRE Trans Inform Theory 4:69–72
Rall W (1955) Experimental monosynaptic input-output relations in the mammalian spinal cord. J Cell Comp Physiol 46:413–437
Reilly DL, Cooper LN (1990) An overview of neural networks: early models to real world systems. In: Zornetzer SF, Davis JL, Lau C (eds) An introduction to neural and electronic networks. Academic Press, San Diego, pp 227–245
Rice SO (1954) Mathematical analysis of random noise. In: Wax N (ed) Selected papers on noise and stochastic processes. Dover, New York, pp 133–294
Rosenblueth A, Wiener N, Pitts W, Ramos JG (1949) A statistical analysis of synaptic excitation. J Cell Comp Physiol 34(2): 173–205
Rumelhart DE, Hinton GE, McClelland JL (1986) A general framework for parallel distributed processing. In: Rumelhart DE McClelland JL (eds) Parallel distributed processing. Explorations in the microstructure of cognition, vol 1. (A Bradford book) MIT Press, Cambridge, Mass, pp 45–76
Siegel RM (1990) Non-linear dynamical system theory and primary visual cortical processing. Physica D 42:385–395
Singer W (1990) Search for coherence: a basic principle of cortical self-organization. Concepts Neurosci 1:1–26
Skarda CA, Freeman WJ (1987) How brains make chaos in order to make sense of the world. Behav Brain Sci 10:161–195
Wise GL, Traganitis AP, Thomas JB (1977) The effect of a memoryless non-linearity on the spectrum of a random process. IEEE Trans Inform Theory IT-23 (1):84–89
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Bedenbaugh, P., Gerstein, G.L. Rectification of correlation by a sigmoid nonlinearity. Biol. Cybern. 70, 219–225 (1994). https://doi.org/10.1007/BF00197602
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00197602