Abstract
Cues are informative signals animals must use to make decisions in order to obtain rewards, usually after intervening temporal delays, typified in the cue-action-reward task. In behavioural experiments the cue is often clearly distinguished from other stimuli, by a salience such as brightness for example, however in the real world animals face the problem of recognizing real cues from among other environmental distracting stimuli. Furthermore once the cue is recognized it must cause the animal to make a certain action to obtain reward. Therefore the animal faces a compound chicken-and-egg problem to obtain reward. First it must recognize the cue and then it must learn that the cue must initiate a certain action. But how can the animal recognize the cue before it has learned the action to obtain reward, since in this initial learning stage the cue is only partially predictive of reward? Here we present a simple neural network model of how animals extract cues from background distractor stimulus, all presented with equal salience, based on successive testing of different stimulus-action allocations over several trials. A stimulus is selected and gated into working memory to drive an action and then reactivated at the end period to be reinforced if correct. If the stimulus is not reinforced over several trials it is suppressed and a different stimulus is selected. If the stimulus is a real cue but it drives the incorrect action, its cue-action allocation is suppressed. This mechanism is enhanced by the property of cue mutual exclusion in trials which also provides a simple model of bottom-up attention and pop-out. The model is based on the cortical and hippocampal projections to the dopamine system through the striatum including a model of salience gated working memory and a reinforcement and punishment system based on dopamine feedback balance. We illustrate the model by numerical simulations of a rat learning to navigate a T-maze and show how it deterministically discovers the correct cue-action allocations.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Schultz, W.: J. Neurophysiol. 80(1), 1–27 (1998)
Wickens, J.: 8, R77–R109 (1997)
Pasupathy, A., Miller, E.K.: Nature 433, 873 (2005)
Kawagoe, R., Takikawa, Y., Hikosaka, O.: J. Neurophysiol. 91(2), 1013–1024 (2004)
Hollerman, J.R., Tremblay, L., Schultz, W.: J. Neurophysiol. 80, 947–963 (1998)
Nicola, S.M., Surmeier, J., Malenka, R.C.: Ann. Rev. Neuroscience. 23, 185 (2000)
Lisman, J.E., Grace, A.A.: Neuron 46, 703 (2005)
Goldman-Rakic, P.S.: Neuron 14(3), 477–485 (1995)
Gruber, A.J., Dayan, P., Gutkin, B.S., Solla, S.A.: J. Comput. Neurosci. 20, 153–166 (2006)
Cohen, J.D., Braver, T.S., Brown, J.W.: Curr. Opin. Neurobiol. 12(2), 223–229 (2002)
Dreher, J.C., Guigon, E., Burnod, Y.: J. Cog. Neurosci. 14(6), 853–865 (2002)
Frank, M.J., Loughry, B., O’Reilly, R.C.: Cog. A. & B. Neurosci. 1(2), 137–160 (2001)
Kiyatkin, E.A., Rebec, G.V.: J. Neurophysiol. 75(1), 142–153 (1996)
Hasselmo, M.E., Eichenbaum, H.: Neural Networks, 18, 1172 (2005)
Wood, E.R., Dudchenko, P.A., Robitsek, R.J., Eichenbaum, H.: Neuron 27, 623–633 (2000)
Kunec, S., Hasselmo, M.E., Kopell, N.: J.Neurophysiol. 94 (2005)
Ferbinteanu, J., Shapiro, M.L.: Neuron 40, 1227–1239 (2003)
Anderson, M.I., Jeffery, K.J.: J. Neurosci. 23, 8827–8835 (2003)
Mulder, A.B., Tabuchi, E., Wiener, S.I.: European J.Neuroscience 19, 1923–1932 (2001)
Barnes, T.D., Kubota, Y., Hu, D., Jin, D.Z., Graybiel, A.M.: Nature 437, 1158 (2005)
Ponzi, A.: Proceedings of IJCNN 2007 (2007)
Ponzi, A.: Neural Networks (to appear, 2008)
Reynolds, J.N.J., Hyland, B.I., Wickens, J.R.: Nature 413, 67 (2001)
Nakahara, H., Amari, S., Hikosaka, O.: Neural Computation 14, 819–844 (2002)
Ponzi, A.: Proceedings of ICCN 2007 (2007)
Ponzi, A.: IEICE Technical Report 103(163), 19–24 (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ponzi, A. (2008). Model of Cue Extraction from Distractors by Active Recall. In: Ishikawa, M., Doya, K., Miyamoto, H., Yamakawa, T. (eds) Neural Information Processing. ICONIP 2007. Lecture Notes in Computer Science, vol 4984. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69158-7_29
Download citation
DOI: https://doi.org/10.1007/978-3-540-69158-7_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69154-9
Online ISBN: 978-3-540-69158-7
eBook Packages: Computer ScienceComputer Science (R0)