Abstract
The Hierarchical Temporal Memory (HTM) framework is a deep learning system inspired by the functioning of the human neocortex. In this paper we investigate the feasibility of this framework by evaluating the performance of one component, the spatial pooler. Using a recently developed implementation, the augmented spatial pooler (ASP), as a single layer feature detector, we test its performance using a standard image classification pipeline. The main contributions of the paper are the implementation and evaluation of modifications to ASP that enable it to form overcomplete representations of the input and to form connections with multiple data channels. Our results show that these modifications significantly improve the utility of ASP, making its performance competitive with more traditional feature detectors such as sparse restricted Boltzmann machines and sparse auto-encoders.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barlow, H.B.: The Ferrier lecture, 1980: Critical limiting factors in the design of the eye and visual cortex. Proceedings of the Royal Society of London. Series B. Biological Sciences 212(1186), 1–34 (1981)
Clark, A.: Whatever next? predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences 36(3), 181–204 (2013)
Coates, A., Ng, A.Y., Lee, H.: An analysis of single-layer networks in unsupervised feature learning. In: International Conference on Artificial Intelligence and Statistics, pp. 215–223 (2011)
Deng, J., Zhang, Z., Marchi, E., Schuller, B.: Sparse autoencoder-based feature transfer learning for speech emotion recognition. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, pp. 511–516 (2013)
Fino, E., Yuste, R.: Dense inhibitory connectivity in neocortex. Neuron 69(6), 1188–1203 (2011)
Friston, K.: The free-energy principle: a unified brain theory? Nature Reviews Neuroscience 11(2), 127–138 (2010)
Hawkins, J., Ahmad, S., Dubinsky, D.: Hierarchical temporal memory including HTM cortical learning algorithms. Tech. rep., Numenta Inc., Palto Alto (2011)
Hawkins, J., Blakeslee, S.: On Intelligence. Henry Holt, New York (2004)
Hawkins, J., George, D.: Hierarchical temporal memory: Concepts, theory and terminology. Tech. rep., Numenta Inc., Palto Alto (2006)
Hebb, D.O.: The organization of behavior: A neuropsychological theory. Wiley, New York (1949)
Hinton, G., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Computation 18(7), 1527–1554 (2006)
Holmgren, C., Harkany, T., Svennenfors, B., Zilberter, Y.: Pyramidal cell communication within local networks in layer 2/3 of rat neocortex. The Journal of Physiology 551(1), 139–153 (2003)
Isaacson, J.S., Scanziani, M.: How inhibition shapes cortical activity. Neuron 72(2), 231–243 (2011)
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Tech. rep., Computer Science Department, University of Toronto, Toronto (2009)
Main, L., Cowley, B., Kneller, A., Thornton, J.: Evaluating sparse codes on handwritten digits. In: Cranefield, S., Nayak, A. (eds.) AI 2013. LNCS, vol. 8272, pp. 396–407. Springer, Heidelberg (2013)
Malkin, R.G., Waibel, A.: Classifying user environment for mobile applications using linear autoencoding of ambient audio. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 5, pp. 509–512 (2005)
Markram, H., Toledo-Rodriguez, M., Wang, Y., Gupta, A., Silberberg, G., Wu, C.: Interneurons of the neocortical inhibitory system. Nature Reviews Neuroscience 5(10), 793–807 (2004)
Mountcastle, V.B.: The columnar organization of the neocortex. Brain 120(4), 701–722 (1997)
Olshausen, B.A.: Principles of image representation in visual cortex. In: The Visual Neurosciences, pp. 1603–1615. MIT Press, Cambridge (2003)
Teh, Y.W., Hinton, G.E.: Rate-coded restricted Boltzmann machines for face recognition. In: Advances in Neural Information Processing Systems, pp. 908–914 (2001)
Thomson, A.M., Lamy, C.: Functional maps of neocortical local circuitry. Frontiers in Neuroscience 1(1), 19–42 (2007)
Thornton, J., Main, L., Srbic, A.: Fixed frame temporal pooling. In: Thielscher, M., Zhang, D. (eds.) AI 2012. LNCS, vol. 7691, pp. 707–718. Springer, Heidelberg (2012)
Thornton, J., Srbic, A.: Spatial pooling for greyscale images. International Journal of Machine Learning and Cybernetics 4(3), 207–216 (2013)
Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Cowley, B., Kneller, A., Thornton, J. (2014). Cortically-Inspired Overcomplete Feature Learning for Colour Images. In: Pham, DN., Park, SB. (eds) PRICAI 2014: Trends in Artificial Intelligence. PRICAI 2014. Lecture Notes in Computer Science(), vol 8862. Springer, Cham. https://doi.org/10.1007/978-3-319-13560-1_57
Download citation
DOI: https://doi.org/10.1007/978-3-319-13560-1_57
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-13559-5
Online ISBN: 978-3-319-13560-1
eBook Packages: Computer ScienceComputer Science (R0)