Abstract
A memory capacity exists for artificial neural networks of associative memory. The addition of new memories beyond the capacity overloads the network system and makes all learned memories irretrievable (catastrophic forgetting) unless there is a provision for forgetting old memories. This article describes a property of associative memory networks in which a number of units are replaced when networks learn. In our network, every time the network learns a new item or pattern, a number of units are erased and the same number of units are added. It is shown that the memory capacity of the network depends on the number of replaced units, and that there exists a optimal number of replaced units in which the memory capacity is maximized. The optimal number of replaced units is small, and seems to be independent of the network size.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anderson JA (1972) A simple neural network generating interactive memory. Math Biosci 14:197–220
Kohonen T (1972) Correlation matrix memories. IEEE Trans Comput C-21:353–359
Nakano K (1972) Associatron: a model of associative memory. IEEE Trans Syst Man Cybern SMC-2:381–388
Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA 79:2554–2558
Amit DJ, Gutfreund H, Sompolinsky H (1985) Storing infinite numbers of patterns in a spun-glass model of neural networks. Phys Rev Lett 55:1530–1533
Nadal JP, Toulouse G, Changeux JP, et al. (1986) Network of formal neurons and memory palimpsests Europhys Lett 1:535–542
Mézard M, Nadal JP, Toulouse G (1986) Solvable models of working memories. J Phys 47:1457–1462
Amari S, Maginu K (1988) Statistical neurodynamics of associative memory. Neural Networks 1:63–73
Hebb DO (1949) The organization of behavior. Wiley, New York
McNaughton BL, Morris RGM (1987) Hippocampal synaptic enhancement and information storage within a distributed system. Trends Neurosci 10:408–415
Nakazawa K, Quirk MC, Chitwood RA, et al. (2002) Requirement for hippocampal CA3 NMDA receptors in associative memory recall. Science 297:211–218
Marr D (1971) Simple memory: a theory for archicortex. Phil Trans R Soc London Ser B 262:23–81
Willshaw D (1990) An assessment of Marr’s theory of the hippocampus as a temporary memory store. Phil Trans R Soc London Ser B 329:205–215
Becker S (2005) A computational principle for hippocampal learning and neurogenesis. Hippocampus 15:722–738
Cecchi GA, Magnasco MO (2005) Computational models of adult neurogenesis. Physica A 356:43–47
Wiskott L, Rasch M, Kempermann G (2006) A functional hypothesis for adult hippocampal neurogenesis: avoidance of catastrophic interference in the dentate gyrus. Hippocampus 16:329–343
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Date, A., Kurata, K. A property of neural networks of associative memory with replacing units. Artif Life Robotics 12, 291–294 (2008). https://doi.org/10.1007/s10015-007-0484-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10015-007-0484-2