Abstract
Neurophysiological experimental results suggest that basal ganglia plays crucial role in action selection while dopamine modifies this process. There are computational models based on these experimental results for action selection. This work focuses on modification of action selection by dopamine release. In the model, a dynamical system is considered for action selection and modification of action selection process is realized by reinforcement learning. The ability of the proposed dynamical system is investigated by bifurcation analysis. Based on the results of this bifurcation analysis, the effect of reinforcement learning on action selection is discussed. The model is implemented on a mobile robot and a foraging task is realized where an exploration in an unfamiliar environment with training in the world is accomplished. Thus, this work fulfills its aim of showing the efficiency of brain-inspired computational models in controlling intelligent agents.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Redgrave, P., Prescott, T.J., Gurney, K.: The basal ganglia: A vertebrate solution to the selection problem? Neuroscience 89, 1009–1023 (1999)
Dayan, P., Daw, N.D.: Decision theory, reinforcement learning and brain. Cognitive, Affective & Behavioral Neuroscience 8(4), 429–453 (2008)
Alexander, G.E., Cruther, M.D., DeLong, M.R.: Basal ganglia-thalamocortical circuits: Parallel substrates for motor, oculomotor, ”prefrontal” and ”limbic” functions. Progress in Brain Research 85, 119–146 (1990)
Haber, S.N.: The reward circuit: Linking primate anatomy and human imaging. Neuropsychopharmacology Reviews 35, 4–26 (2010)
Denizdurduran, B., Sengor, N.S.: A Realization of Goal-Directed Behavior, Implementing a Robot Model Based on Cortico-Striato-Thalamic Circuits. In: Proceedings of The 4th International Conference on Agents and Artificial Intelligence, pp. 289–294 (2012)
Prescott, T.J., Montes-Gonzales, F.M., Gurney, K., Humpries, M.D., Redgrave, P.: A Robot Model of the Basal Ganglia: Behaviour and Intrinsic Processing. Neural Networks, 1–31 (2006)
Houk, J.C., Bastianen, C., Fansler, D., Fishbach, A., Fraser, D., Reber, P.J., Roy, S.A., Simo, L.S.: Action selection and refinement in subcortical loops through basal ganglia and cerebellum. Phil. Trans. R. Soc. B, 29 362(1485), 1573–1583 (2007)
Schultz, W., Dayan, P., Montague, P.R.: A Neural Substrate of Prediction and Reward. Science 275, 1593–1599 (1997)
Frank, M.J.: Computational models of motivated action selection in corticostriatal circuits. Current Opinion in Neurobiology 21, 381–386 (2011)
Kuznetsov, Y.A.: Elements of Bifurcation Theory. In: Marsden, J.E., Sirovich, L. (eds.) Applied Mathematical Sciences, 2nd edn., vol. 112 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Denizdurduran, B., Sengor, N.S. (2012). Learning How to Select an Action: A Computational Model. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds) Artificial Neural Networks and Machine Learning – ICANN 2012. ICANN 2012. Lecture Notes in Computer Science, vol 7552. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33269-2_60
Download citation
DOI: https://doi.org/10.1007/978-3-642-33269-2_60
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33268-5
Online ISBN: 978-3-642-33269-2
eBook Packages: Computer ScienceComputer Science (R0)