Abstract
We examine the performance of Hebbian-like attractor neural networks, recalling stored memory patterns from their distorted versions. Searching for an activation (firing-rate) function that maximizes the performance in sparsely connected low-activity networks, we show that the optimal activation function is a threshold-sigmoid of the neuron's input field. This function is shown to be in close correspondence with the dependence of the firing rate of cortical neurons on their integrated input current, as described by neurophysiological recordings and conduction-based models. It also accounts for the decreasing-density shape of firing rates that has been reported in the literature.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abeles M (1991) Corticonics: neural circuits of the cerebral cortex. Cambridge University Press, Cambridge
Abeles M, Vaadia E, Bergman H (1990) Firing patterns of single units in the prefrontal cortex and neural network models. Network 1:13–25
Amit DJ (1989) Modeling brain function: the world of attractor neural networks. Cambridge University Press, Cambridge
Amit DJ, Tsodyks MV (1991) Quantitative study of attractor neural network retrieving at low spike rates. I. Substrate-spikes, rates and neuronal gain. Network 2:259–273
Avoli M, Olivier A (1989) Electrophysiological properties and synaptic responses in the deep layers of the human epileptogenic neocortex in vitro. J Neurophysiol 61:589–606
Derrida B, Gardner E, Zippelius A (1987) An exactly solvable asymmetric neural network model. Europhys Lett 4:167–173
Doya K, Selverston AI (1994) Dimension reduction of biological neuron models by artificial neural networks. Neural Comput 6:696–717
Durbin R, Mitchison G (1990) A dimension reduction framework for understanding cortical maps. Nature 343:644–647
Englisch H, Xiao Y (1991) Neural networks as perpetual information generators. Phys Rev A 44:1382–1385
Ermentrout B (1994) Reduction of conduction-based models with slow synpases to neural nets. Neural Comput 6:679–695
Evans MR (1989) Random dilution in a neural network for biased patterns. J Phys A: Math Gen 22:2103–2118
Gustafsson B, Wigstrom H (1981) Shape of frequency-current curves in Cal pyramidal cells in the hippocampus. Brain Res 223:417–421
Hinton GE, Nowlan SJ (1990) The bootstrap Widrow-Hoff rule as a clusterformation algorithm. Neural Comput 2:355–362
Hopfield JJ (1984) Neurons with graded response have collective computational properties like those of two-state neurons. Proc Natl Acad Sci USA 81:3088
Kinzel W (1985) Learning and pattern recognition in spin glass models. Z Phys. B60:205–213
Kononenko I (1989) Bayesian neural networks. Biol Cybern 61:361–370
Lansner A, Ekeberg O (1989) A one-layer feedback artificial neural network with a Bayesian learning rule. Int J Neural Syst 1:77–87
Mason A, Larkman A (1990) Correlations between morphology and electrophysiology of pyramidal neurons in slices of rat visual cortex. I. Electrophysiology. J Neurosci 10:1415–1428
Meilijson I, Ruppin E (1993) History-dependent attractor neural networks. Network 4:195–221
Meilijson I, Ruppin E (1994) Optimal signalling in attractor neural networks. Network 5:277–298
Miyashita Y (1988) Neuronal correlate of visual associative long-term memory in the primate temporal cortex. Nature 335:817–820
Miyashita Y, Chang HS (1988) Neuronal correlate of pictorial short-term memory in the primate temporal cortex. Nature 331:68–71
Ruppin E, Schwartz E, Yeshurun Y (1993) Examining the volume-efficiency of the cortical architecture in a multi-processor network model. Biol Cybern 70:89–94
Treves A, Rolls ET (1991) What determines the capacity of autoassociative memories in the brain? Network 2:371–397
Tsodyks MV, Feigel'man MV (1988) The enhanced storage capacity in neural networks with low activity level. Europhys Lett 6:101–105
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Meilijson, I., Ruppin, E. Optimal firing in sparsely-connected low-activity attractor networks. Biol. Cybern. 74, 479–485 (1996). https://doi.org/10.1007/BF00209419
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00209419