Abstract
Conditional visuo-motor learning consists in learning by trial and error to associate visual cues with correct motor responses, that have no direct link. Converging evidence supports the role of a large brain network in this type of learning, including the prefrontal and the premotor cortex, the basal ganglia (BG) and the hippocampus. In this paper we focus on the role of a major structure of the BG, the striatum. We first present behavioral results and electrophysiological data recorded from this structure in monkeys engaged in learning new visuo-motor associations. Visual stimuli were presented on a video screen and the animals had to learn, by trial and error, to select the correct movement of a joystick, in order to receive a liquid reward. Behavioral results revealed that the monkeys used a sequential strategy, whereby they learned the associations one by one although they were presented randomly. Human subjects, tested on the same task, also used a sequential strategy. Neuronal recordings in monkeys revealed learning-related modulations of neural activity in the striatum. We then present a mathematical model inspired by viability theory developed to implement the use of strategies during learning. This model complements existing models of the BG based on reinforcement learning (RL), which do not take into account the use of strategies to reduce the dimension of the learning space.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alexander GE, DeLong MR, Strick PL (1986) Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci 9:357–381
Aubin J-P (1991) Viability theory. Birkhauser, Boston
Aubin J-P (1996) Neural networks and qualitative physics: a viability approach. University Press, Cambridge
Aubin J-P (2001) Viability kernels and capture basins of sets under differential inclusions. SIAM J Control 40:853–881
Aubin J-P, Frankowska H (1996) The viability kernel algorithm for computing value functions of infinite horizon optimal control problems. J Math Anal Appl 201:555–576
Bar-Gad I, Morris G, Bergman H (2003) Information processing, dimensionality reduction and reinforcement learning in the basal ganglia. Prog Neurobiol 71:439–473
Barto A (1995) Adaptive critics and the basal ganglia. In: Houk JC, Davis JL, Beiser DG (eds) Models of information processing in the basal ganglia. MIT Press, Cambridge, pp 215–232
Boussaoud D, Wise SP (1993a) Primate frontal cortex: neuronal activity following attentional versus intentional cues. Exp Brain Res 95:15–27
Boussaoud D, Wise SP (1993b) Primate frontal cortex: effects of stimulus and movement. Exp Brain Res 95:28–40
Boussaoud D, Kermadi I (1997) The primate striatum: neuronal activity in relation to spatial attention versus motor preparation. Eur J Neurosci 9:2152–2168
Brasted PJ, Wise SP (2004a) Comparison of learning-related neuronal activity in the dorsal premotor cortex and striatum. Eur J Neurosci 19(3):721–740
Brasted PJ, Wise SP (2004b) The arbitrary mapping of sensory inputs to voluntary and involuntary movement: learning-dependant activity in the motor cortex and other telencephalic networks. In: Riehle A, Vaadia E (eds) Motor cortex in voluntary movements. pp 259–293
Bussey TJ, Wise SP, Murray EA (2001) The role of ventral and orbital prefrontal cortex in conditional visuomotor learning and strategy use in rhesus monkeys (Macaca mulatta). Behav Neurosci 115:971–982
Canavan AGM, Nixon PD, Passingham RE (1989) Motor learning in monkeys (Macaca fascicularis) with lesions in motor thalamus. Exp Brain Res 77:113–126
Cardaliaguet P, Quincampoix M, Saint-Pierre P (1999) Set-valued numerical methods for optimal control and differential games, in Stochastic and differential games: theory and numerical methods. Annals of the International Society of Dynamical Games, Birkhauser, pp 177–247
Doya K (1999) What are the computations of the cerebellum, the basal ganglia, and the cerebral cortex. Neural Netw 12:961–974
Doya K (2000) Reinforcement learning in continuous time and space. Neural Comput 12:219–245
Fiorillo CD, Tobler PN, Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299:1898–1902
Frankowska H, Quincampoix M (1991) Viability kernels of differential inclusions with constraints: algorithm and applications. J Math Syst Estim Control 1:371–388
Genovesio A, Brasted PJ, Mitz AR, Wise SP (2005) Prefrontal cortex activity related to abstract response strategies. Neuron 47(2):307–20
Hadj-Bouziane F, Boussaoud D (2003) Activity of neurons in the striatum during learning of arbitrary visuo-motor associations. Exp Brain Res 18:269–277
Hadj-Bouziane F, Meunier M, Boussaoud D (2003) Conditional visuo-motor learning in primates: a key role for the basal ganglia. J Physiol Paris 97:567–579
Hadj-Bouziane F, Benatru I, Klinger H, Brousolle E, Boussaoud D, Meunier M (2004) Conditional associative learning in advanced Parkinson’s disease. Soc Neurosci Abstr 710:4
Halsband U, Freund HJ (1990) Premotor cortex and conditional motor learning in man. Brain 113:207–222
Hollerman JR, Tremblay L, Schultz W (1998) Influence of reward expectation on behavior-related neuronal activity in primate striatum. J Neurophysiol 80:947–963
Houk JC, Adams JL, Barto AG (1995) A model of how the basal ganglia generate and use reward signals that predict reinforcement. In: Houk JC, Davis JL, Beiser DG (eds) Models of information processing in the basal ganglia. MIT Press, Cambridge, pp 249–270
Joel D, Weiner I (1994) The organization of the basal ganglia-thalamocortical circuits: open interconnected rather than closed segregated. Neuroscience 63:363–379
Kurata K, Hoffman DS (1994) Differential effects of muscimol microinjection into dorsal and ventral aspects of the premotor cortex of monkeys. J Neurophysiol 71:1151–1164
Ljungberg T, Apicella P, Schultz W (1992) Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67:145–163
Nixon PD, McDonald KR, Gough PM, Alexander IH, Passingham RE (2004) Cortico-basal ganglia pathways are essential for the recall of well-established visuomotor associations. Eur J Neurosci 20(11):3165–78
Parent A (1990) Extrinsic connections of the basal ganglia. Trends Neurosci 13:254–258
Parent A, Hazrati LN (1995) Functional anatomy of the basal ganglia. I. The cortico-basal gangliathalamo-cortical loop. Brain Res Brain Res Rev 20:91–127
Passingham RE (1993) The frontal lobe and voluntary action. Oxford University Press, Oxford
Passingham RE, Toni I, Rushworth MF (2000) Specialization within the prefrontal cortex: the ventral prefrontal cortex and associative learning. Exp Brain Res 133:103–113
Petrides M (1985) Conditional learning and the primate frontal cortex. In: Perecman E (ed) The frontal lobe revisited. IBRN press, New York, pp 91–108
Prensa L, Cossette M, Parent A (2000) Dopaminergic innervation of human basal ganglia. J Chem Neuroanat 20:207–213
Saint-Pierre P (2001) Approximation of viability kernels and capture basin for hybrid systems. In: Martins de Carvalho JL (ed) Proceedings of the European Control Conference ECC 2001, pp 2776–2783
Saint-Pierre P (2002) Hybrid kernels and capture basins for impulse constrained systems, Communication at International Conference HSCC02, Stanford
Schultz W (2002) Getting formal with dopamine and reward. Neuron 36:241–263
Schultz W, Apicella P, Ljungberg T, Romo R, Scarnati E (1993) Reward-related activity in the monkey striatum and substantia nigra. Prog Brain Res 99:227–235
Smith Y, Kieval JZ (2000) Anatomy of the dopamine system in the basal ganglia. Trends Neurosci 23(Suppl 10):S28–S33
Sutton R, Barto A (1998) Reinforcement learning: an introduction. MIT Press, Cambridge
Toni I, Rowe J, Stephan KE, Passingham RE (2002) Changes of cortico-striatal effective connectivity during visuomotor learning. Cereb Cortex 12:1040–1047
Tremblay L, Hollerman JR, Schultz W (1998) Modifications of reward expectation-related neuronal activity during learning in primate striatum. J Neurophysiol 80:964–977
Wise SP, Murray EA (1999) Role of the hippocampal system in conditional motor learning: mapping antecedents to action. Hippocampus 9:101–117
Wise SP, Murray EA (2000) Arbitrary associations between antecedents and actions. Trends Neurosci 23(6):271–276
Wise SP, Murray EA, Gerfen CR (1996a) The frontal cortex basal ganglia system in primates. Crit Rev Neurobiol 10:317–356
Wise SP, di Pellegrino G, Boussaoud D (1996b) The premotor cortex and nonstandard sensorimotor mapping. Can J Physiol Pharmacol 74:469–482
Acknowledgements
This work was supported by the French Ministry of Research (ACI Cognition et traitement de l’information for HF, and ACI Neurosciences Integratives et computationnelles for DB), Interdisciplinary Program TCAN of CNRS, and a Bettencourt-Schuller foundation fellowship for FHB.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Richard Walker
Rights and permissions
About this article
Cite this article
Hadj-Bouziane, F., Frankowska, H., Meunier, M. et al. Conditional visuo-motor learning and dimension reduction. Cogn Process 7, 95–104 (2006). https://doi.org/10.1007/s10339-005-0028-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10339-005-0028-4