Abstract
A connectionist model of cortico-striato-thalamic loops unifying learning and action selection is proposed. The aim in proposing the connectionist model is to develop a simple model revealing the mechanisms behind the cognitive process of goal directed behaviour rather than merely obtaining a model of neural structures. In the proposed connectionist model, the action selection is realized by a non-linear dynamical system, while learning that modifies the action selection is realized similar to actor-critic model of reinforcement learning. The task of sequence learning is solved with the proposed model to make clear how the model can be implemented.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alexander, G.E., Crutcher, M.D.: Functional architecture of basal ganglia circuits:Neural substates of parallel processing. TINS 13, 266–270 (1990)
Crutcher, M.D., Alexander, G.E.: Basal ganglia-thalamacortical circuits:Parallel substrates for motor, oculomotor, prefrontal and limbic functions. Progress in Brain Research 85, 119–146 (1990)
Heimer, L.: A new anatomical frameworkfor neuropsychiatric disorders and drug abuse. Am. J. Psychiatry 160, 1726–1739 (2003)
Fudge, J.L., Haber, S.N., McFarland, N.R.: Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. The Jour. Neuroscience 20, 2369–2382 (2000)
Haber, N.S., Kim, K.S., Mailly, P., Calzavara, R.: Reward-related cortical inputs definea large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. The Jour. Neuroscience 26, 8368–8376 (2006)
Ridderinkhof, K.R., van den Wildenberg, W.P.M., Segalowitz, S.J., Carter, C.S.: Neurocognitive Mechanisms of Cognitive Control: The Role of Prefrontal Cortex in Action Selection, Response Inhibition. Performance Monitoring and Reward-Based Learning, Brain and Cognition 56, 129–140 (2004)
Taylor, N.R., Taylor, J.G.: Hard-wired Models of Working Memory and Temporal Sequence Storage and Generation. Neural Networks 13, 201–224 (2000)
Taylor, J.G., Taylor, N.R.: Analysis of Recurrent Cortico-Basal Ganglia-Thalamic Loops for Working Memory. Biological Cybernetics 82, 415–432 (2000)
Karabacak, O., Sengor, N.S.: A Dynamical Model of a Cognitive Function: Action Selection. In: 16th IFAC World Congress, Prague (2005)
Karabacak, O., Sengor, N.S.: A computational model for the effect of dopamine on action selection during Stroop test. In: ICANN 2006, Athens (2006)
Schultz, W., Dayan, P., Montague, P.R.: A Neural Substrate of Prediction and Reward. Science 275, 1593–1599 (1997)
Suri, R.E., Schultz, W.: Learning of Sequential Movements by Neural Network Model with Dopamine-Like Reinforcement Signal. Exp. Brain Res. 121, 350–354 (1998)
Holyrod, C.B., Coles, M.G.H.: Neural basis of human error processing: reinforcement learning, dopamine, and error related negativity. Psychological Review 4, 679–709 (2002)
Sutton, R.S., Barto, A.G.: Reinforcement Learning (2nd printing), A Bradford Book. MIT Press, Cambridge (1998)
Berns, G.S., Sejnowski, T.J.: A Model of Basal Ganglia Function Unifying Reinforcement Learning and Action Selection. In: Proc. Joint Symp. Neural Comp., California (1994)
Suri, R.E., Bargas, J., Arbib, M.A.: Modeling functions of striatal dopamine modulation in learning and planning. Neuroscience 103, 65–85 (2001)
Dayan, P., Balleine, B.W.: Reward, Motivation and Reinforcement Learning. Neuron 36, 285–298 (2002)
Montague, P.R., Hyman, S.E., Cohen, J.D.: Computational Roles for Dopamine in Behavioural Control. Nature 431, 760–767 (2004)
Joel, D.A., Niv, Y., Ruppin, E.: Actor-Critic models of the basal ganglia: new anatomical and computational perspectives. Neural Networks 15, 535–547 (2002)
Gurney, K., Prescott, T.J., Redgrave, P.: A Computational Model of Action Selection in the Basal Ganglia I: A New Functional Anatomy. Biological Cybernetics 84, 401–410 (2001)
Gurney, K., Prescott, T.J., Redgrave, P.: A Computational Model of Action Selection in the Basal Ganglia II: Analysis and Simulation of Behaviour. Biological Cybernetics 84, 411–423 (2001)
Haruno, M., Kawato, M.: Heterarchical reinforcement learning model for integration of multiple cortico-striatal loops:fMRI examination in stimulus-action-reward association learning. Neural Networks 19, 1242–1254 (2006)
Graybiel, A.M.: Neurotransmitters and Neuromodulators in the Basal Ganglia. TINS 13, 244–254 (1990)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Şengör, N.S., Karabacak, ”., Steinmetz, U. (2008). A Computational Model of Cortico-Striato-Thalamic Circuits in Goal-Directed Behaviour. In: Kůrková, V., Neruda, R., Koutník, J. (eds) Artificial Neural Networks - ICANN 2008. ICANN 2008. Lecture Notes in Computer Science, vol 5164. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87559-8_34
Download citation
DOI: https://doi.org/10.1007/978-3-540-87559-8_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87558-1
Online ISBN: 978-3-540-87559-8
eBook Packages: Computer ScienceComputer Science (R0)