Abstract
Strong experimental evidence suggests that cortical memory traces are consolidated during off-line memory reprocessing that occurs in the off-line states of sleep or waking rest. It is unclear, what plasticity mechanisms are involved in this process and what changes are induced in the network in the off-line regime. Here, we examine a hierarchical recurrent neural network that performs unsupervised learning on natural face images of different persons. The proposed network is able to self-generate memory replay while it is decoupled from external stimuli. Remarkably, the recognition performance is tremendously boosted after this off-line regime specifically for the novel face views that were not shown during the initial learning. This effect is independent of synapse-specific plasticity, relying completely on homeostatic regulation of intrinsic excitability. Comparing a purely feed-forward network configuration with the full version reveals a substantially stronger boost in recognition performance for the fully recurrent network architecture after the off-line regime.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Jitsev, J., von der Malsburg, C.: Experience-driven formation of parts-based representations in a model of layered visual memory. Front. Comput. Neurosci. 3, 15 (2009)
Lücke, J.: Receptive field self-organization in a model of the fine structure in V1 cortical columns. Neural Comput. 21(10), 2805–2845 (2009)
Fries, P., Nikolić, D., Singer, W.: The gamma cycle. Trends Neurosci. 30(7), 309–316 (2007)
Jitsev, J.: On the self-organization of a hierarchical memory for compositional object representation in the visual cortex. PhD thesis, Goethe University Frankfurt, Frankfurt Institute for Advanced Studies (November 2010)
Martinez, A.M., Benavente, R.: The AR face database. Technical Report 24, CVC Technical Report 24 (June 1998)
Jitsev, J., von der Malsburg, C.: Off-line memory reprocessing following on-line unsupervised learning strongly improves recognition performance in a hierarchical visual memory. In: International Joint Conference on Neural Networks (IJCNN), pp. 3123–3130. IEEE (July 2010)
Rasch, B., Born, J.: Maintaining memories by reactivation. Curr. Opin. Neurobiol. 17(6), 698–703 (2007)
Diekelmann, S., Born, J.: The memory function of sleep. Nat. Rev. Neurosci. 11(2), 114–126 (2010)
Zhang, W., Linden, D.J.: The other side of the engram: experience-driven changes in neuronal intrinsic excitability. Nat. Rev. Neurosci. 4(11), 885–900 (2003)
Karmarkar, U.R., Buonomano, D.V.: Different forms of homeostatic plasticity are engaged with distinct temporal profiles. Eur. J. Neurosci. 23(6), 1575–1584 (2006)
Axmacher, N., Haupt, S., Fernández, G., Elger, C.E., Fell, J.: The role of sleep in declarative memory consolidation–direct evidence by intracranial EEG. Cereb. Cortex 18(3), 500–507 (2008)
Lahl, O., Wispel, C., Willigens, B., Pietrowsky, R.: An ultra short episode of sleep is sufficient to promote declarative memory performance. J. Sleep Res. 17, 3–10 (2008)
Tononi, G., Cirelli, C.: Sleep and synaptic homeostasis: A hypothesis. Brain Res. Bull. 62(2), 143–150 (2003)
Olcese, U., Esser, S.K., Tononi, G.: Sleep and synaptic renormalization: A computational study. J. Neurophysiol. 104(6), 3476–3493 (2010)
Tononi, G., Cirelli, C.: Sleep and the price of plasticity: from synaptic and cellular homeostasis to memory consolidation and integration. Neuron 81(1), 12–34 (2014)
Crick, F., Mitchison, G.: The function of dream sleep. Nature 304(5922), 111–114 (1983)
Baran, B., Wilson, J., Spencer, R.C.: REM-dependent repair of competitive memory suppression. Exp. Brain Res. 203(2), 471–477 (2010)
Cirelli, C., Tononi, G.: Is sleep essential? PLoS Biol. 6(8), e216 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Jitsev, J. (2014). Self-generated Off-line Memory Reprocessing Strongly Improves Generalization in a Hierarchical Recurrent Neural Network. In: Wermter, S., et al. Artificial Neural Networks and Machine Learning – ICANN 2014. ICANN 2014. Lecture Notes in Computer Science, vol 8681. Springer, Cham. https://doi.org/10.1007/978-3-319-11179-7_83
Download citation
DOI: https://doi.org/10.1007/978-3-319-11179-7_83
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11178-0
Online ISBN: 978-3-319-11179-7
eBook Packages: Computer ScienceComputer Science (R0)