Abstract
The neocognitron is a multi-layered convolutional network that can be trained to recognize visual patterns robustly. This paper discusses a new neocognitron, which uses the add-if-silent rule for training intermediate layers and the method of interpolating-vector for classifying patterns at the highest stage of the hierarchical network. By the add-if-silent rule, a new cell is generated when all postsynaptic cells are silent. The generated cell learns the activity of the presynaptic cells in one-shot, and its input connections will never be modified afterward. Thus the training process is very simple, and does not require time-consuming calculation such as the gradient descent process. This paper analyzes how the size of training set affects the performance of the neocognitron and show that the add-if-silent rule can produce feature-extracting cells efficiently even with a small number of training patterns.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Hinton, G.E., Osindero, S., Teh, Y.: A Fast Learning Algorithm for Deep Belief Nets. Neural Computation 18, 1527–1554 (2006)
Bengio, Y.: Learning deep architectures for AI. Foundations and Trends in Machine Learning 2(1), 1–127 (2009)
Fukushima, K.: Training Multi-layered Neural Network Neocognitron. Neural Networks 40, 18–31 (2013)
Fukushima, K.: Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position. Biol. Cybernetics 36(4), 193–202 (1980)
Fukushima, K.: Artificial Vision by Multi-layered Neural Networks: Neocognitron and its Advances. Neural Networks 37, 103–119 (2013)
Fukushima, K.: One-shot Learning with Feedback for Multi-layered Convolutional Network. In: Wermter, S., Weber, C., Duch, W., Honkela, T., Koprinkova-Hristova, P., Magg, S., Palm, G., Villa, A.E.P. (eds.) ICANN 2014. LNCS, vol. 8681, pp. 291–298. Springer, Heidelberg (2014)
Fukushima, K.: Interpolating Vectors for Robust Pattern Recognition. Neural Networks 20(8), 904–916 (2007)
ETL1 database, http://projects.itri.aist.go.jp/etlcdb/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Fukushima, K. (2014). Add-if-Silent Rule for Training Multi-layered Convolutional Network Neocognitron. In: Loo, C.K., Yap, K.S., Wong, K.W., Teoh, A., Huang, K. (eds) Neural Information Processing. ICONIP 2014. Lecture Notes in Computer Science, vol 8834. Springer, Cham. https://doi.org/10.1007/978-3-319-12637-1_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-12637-1_10
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-12636-4
Online ISBN: 978-3-319-12637-1
eBook Packages: Computer ScienceComputer Science (R0)