Abstract
Attention is both ubiquitous throughout and key to our cognitive experience. It has been shown to filter out mundane stimuli, while simultaneously communicating specific stimuli from the lowest levels of perception through to the highest levels of cognition. In this paper we present a connectionist system with mechanisms that produce both exogenous (bottom-up) and endogenous (top-down) attention. The foundational algorithm of our system is the Temporal Pooler (TP), a neocortically inspired algorithm that learns and predicts temporal sequences. We make a number of modifications to the Temporal Pooler and place it in a framework which is inspired by predictive coding. We use a novel technique in which feedback connections elicit endogenous attention by disrupting the learned representations of attended sequences. Our experiments show that this approach successfully filters attended stimuli and suppresses unattended stimuli.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ba, J., Salakhutdinov, R.R., Grosse, R.B., Frey, B.J.: Learning wake-sleep recurrent attention models. In: Advances in Neural Information Processing Systems, pp. 2575–2583 (2015)
Bengio, Y., Lee, D.H., Bornschein, J., Lin, Z.: Towards biologically plausible deep learning (2015). arXiv preprint: arXiv:1502.04156
Broadbent, D.E.: Perception and Communication. Oxford University Press, Oxford (1958)
Clark, A.: Surfing Uncertainty: Prediction, Action, and the Embodied Mind. Oxford University Press, Oxford (2015)
Denil, M., Bazzani, L., Larochelle, H., de Freitas, N.: Learning where to attend with deep architectures for image tracking. Neural Comput. 24(8), 2151–2184 (2012)
Desimone, R., Duncan, J.: Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18(1), 193–222 (1995)
Feldman, H., Friston, K.J.: Attention, uncertainty, and free-energy. Front. Hum. Neurosci. 4, 215 (2010)
Friston, K.: The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 11(2), 127–138 (2010)
Hawkins, J., Ahmad, S., Dubinsky, D.: Hierarchical temporal memory including HTM cortical learning algorithms. Technical report, Numenta, Inc., Palto Alto (2010)
Hawkins, J., Blakeslee, S.: On Intelligence. Macmillan, New York (2007)
Itti, L., Baldi, P.F.: Bayesian surprise attracts human attention. In: Advances in Neural Information Processing Systems, pp. 547–554 (2005)
Kneller, A., Thornton, J.: Distal dendrite feedback in hierarchical temporal memory. In: Proceedings of the 2015 International Joint Conference on Neural Networks (2015)
Lachter, J., Forster, K.I., Ruthruff, E.: Forty-five years after broadbent (1958): still no identification without attention. Psychol. Rev. 111(4), 880 (2004)
Larochelle, H., Hinton, G.E.: Learning to combine foveal glimpses with a third-order Boltzmann machine. In: Advances in Neural Information Processing Systems, pp. 1243–1251 (2010)
Lavin, A., Ahmad, S.: Evaluating real-time anomaly detection algorithms - the numenta anomaly bench. In: 14th International Conference on Machine Learning and Applications (2015)
McCall, R., Franklin, S.: Cortical learning algorithms with predictive coding for a systems-level cognitive architecture. In: Second Annual Conference on Advances in Cognitive Systems Poster Collection, pp. 149–166 (2013)
Moran, J., Desimone, R.: Selective attention gates visual processing in the extrastriate cortex. Science 229(4715), 782–784 (1985)
Mountcastle, V.B.: The columnar organization of the neocortex. Brain 120(4), 701–722 (1997)
Rao, R.P., Ballard, D.H.: Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2(1), 79–87 (1999)
Rao, R.P., Ballard, D.H.: Probabilistic models of attention based on iconic representations and predictive coding. In: Neurobiology of Attention, pp. 553–561 (2004)
Reynolds, J.H., Chelazzi, L.: Attentional modulation of visual processing. Annu. Rev. Neurosci. 27, 611–647 (2004)
Spratling, M.W.: Predictive coding as a model of biased competition in visual attention. Vis. Res. 48(12), 1391–1408 (2008)
Stollenga, M.F., Masci, J., Gomez, F., Schmidhuber, J.: Deep networks with internal selective attention through feedback connections. In: Advances in Neural Information Processing Systems, pp. 3545–3553 (2014)
Tang, Y., Srivastava, N., Salakhutdinov, R.R.: Learning generative models with visual attention. In: Advances in Neural Information Processing Systems, pp. 1808–1816 (2014)
Wang, Q., Zhang, J., Song, S., Zhang, Z.: Attentional neural network: feature selection using cognitive feedback. In: Advances in Neural Information Processing Systems, pp. 2033–2041 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Cowley, B., Thornton, J. (2017). Feedback Modulated Attention Within a Predictive Framework. In: Wagner, M., Li, X., Hendtlass, T. (eds) Artificial Life and Computational Intelligence. ACALCI 2017. Lecture Notes in Computer Science(), vol 10142. Springer, Cham. https://doi.org/10.1007/978-3-319-51691-2_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-51691-2_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-51690-5
Online ISBN: 978-3-319-51691-2
eBook Packages: Computer ScienceComputer Science (R0)