Abstract
A novel neural network model is presented that learns by trial-and-error to reproduce complex sensory-motor sequences. One subnetwork, corresponding to the prefrontal cortex (PFC), is responsible for generating unique patterns of activity that represent the continuous state of sequence execution. A second subnetwork, corresponding to the striatum, associates these state-encoding patterns with the correct response at each point in the sequence execution. From a neuroscience perspective, the model is based on the known cortical and subcortical anatomy of the primate oculomotor system. From a theoretical perspective, the architecture is similar to that of a finite automaton in which outputs and state transitions are generated as a function of inputs and the current state. Simulation results for complex sequence reproduction and sequence discrimination are presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alexander GE, DeLong MR, Strick PL (1986) Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci 9:357–381
Andersen RA, Asanuma C, Cowan MW (1985) Callosal and prefrontal associational projecting cell populations in area 7a of the macaque monkey: a study using retrogradely transported flourescent dyes. J Comp Neurol 232:443–455
Barash S, Bracewell RM, Fogassi L, Gnadt JW, Andersen RA (1991) Saccade-related activity in the lateral intraparietal area II. Spatial properties. J Neurophysiol 66:1109–1124
Barone P, Joseph J-P (1989) Prefrontal cortex and spatial sequencing in macaque moneky. Exp Brain Res 78:447–464
Barto AG (1990) Connectionist learning for control: an overview. In: Miller WT, Sutton RS, Werbos PJ (eds) Neural Networks for Control. MIT Press, Cambridge MA, pp 1–58
Calabresi P, Maj R, Pisani A, Mercuir NB, Bernardi G (1992) Long-term synaptic depression in the striatum: physiological and pharmacological characterization. J Neurosci 12:4224–4233
Carlsson M, Carlsson A (1990) Interactions between glutamatergic and monoaminergic systems within the basal ganglia — indications for schizophrenia and Parkinson's disease. Trends Neurosci 13:272–276
Chevalier G, Vacher S, Deniau JM, Desban M (1985) Disinhibition as a basic process in the expression of striatal functions. I. The striato-nigral influence on the tecto-spinal tecto-diencephalic neurons. Brain Res 334:215–226
Dehaene S, Changeux J-P, Nadal J-P (1987) Neural networks that learn temporal sequences by preselection. Proc Natl Acad Sci 84:2727–2731
Dominey PF (1993) Models of spatial accuracy and conditional behavior in oculomotor control. PhD Thesis, Dept of Computer Science, University of Southern California
Dominey PF, Arbib MA (1992) A cortico-subcortical model for generation of spatially accurate sequential saccades. Cerebral Cortex 2:153–175
Dominey PF, Arbib MA, Joseph JP (1995) A model of corticostriatal plasticity for learning oculomotor associations and sequences. J Cog Neuroscience 7:311–336
Goldman-Rakic PS (1987) Circuitry of primate prefrontal cortex and regulation of behavior by representational memory. In: Vernon B Mountcastle (eds) Handbook of physiology. The Nervous system. Vol. V, Chap 9. 9: 373–417
Herz A, Sulzer B, Kühn R, van Hemmen JL (1989) Hebbian learning reconsidered: representation of static and dynamic objects in associative neural nets. Biol Cybern 60:457–467
Hikosaka O, Sakamoto M, Usui S (1989) Functional properties of monkey caudate neurons. I. Activities related to saccadic eye movements. J Neurophysiol 61:780–798
Keele SW, Jennings P, Jones S, Caulton D, Cohen A (1995) On the modularity of sequence representation. J Motor Behavior 27:17–30
Kermadi I, Jurquet Y, Arzi M, Joseph J-P (1993) Neural activity in the caudate nucleus of monkeys during spatial sequencing. Exp Brain Res 94:352–356
Kühn R, van Hemmen JL (1992) Temporal association. In: Domanay E, Hemmen van JL, Schulten K (eds) Physics of neural networks. Springer, Berlin Heidelberg New York, pp 213–280
Ljungberg T, Apicella P, Schultz W (1991) Responses of monkey midbrain dopamine neurons during delayed alternation performance. Brain Res 567:337–341
Mercuri M, Bernardi G, Calabresi P, Cotugno A, Levi G, Stanzione P (1985) Dopamine decreases cell excitability in rat striatal neurons by pre- and postsynaptic mechanisms. Brain Res 385: 110–121
Munoz DP, Wurtz RH (1992) Role of the rostral superior colliculus in active visual fixation on execution of express saccades. J Neurophysiol 67:1000–1002
Reading PJ, Dunnet SB, Robins TW (1991) Dissociable roles of the ventral, medial and lateral striatum on the acquisition and performance of a complex visual stimulus-response habit. Behav Brain Res 45:147–161
Robins TW, Giardini V, Jones GH, Reading PJ, Sahakian BJ (1990) Effects of dopamine depletion from the caudate-putamen and nucleus accumbes septi on the acquisition and performance of a conditional discrimination task. Behav Brain Res 38:243–261
Rolls ET, Williams GV (1987) Sensory and movement-related neuronal activity in different regions of the primate striatum. In: Schneider JS, Lidsky TI (eds) Basal ganglia and motor behavior. Hans Huner, Toronto, pp 37–59
Schlag J, Schlag-Rey M (1984) Visuomotor functions of central thalamus in monkey. II. Unit activity related to visual events, targeting, and fixation. J Neurophysiol 51:1175–1195
Segraves M, Goldberg ME (1987) Functional properties of corticotectal neurons in the monkey's frontal eye field. J Neurophysiol 58:1387–1419
Selemon LD, Goldman-Rakic PS (1985) Longitudinal topography and interdigitation of corticostriatal projections in the rhesus moneky. J Neurosci 5:776–794
Sparks DL (1986) Translation of sensory signals into commands for control of saccadic eye movements: role of primate superior colliculus. Physiol Rev 66:118–171
Stadler MA (1995) The role of attention in implicit learning. J Exp Psychology: Learning, Mem, Cognition (in press)
Stanton GB, Goldberg ME, Bruce CJ (1988a) Frontal eye field efferents in the macaque monkey. I. Subcortical pathways and topography of striatal and thalamic terminal fields. J Comp Neurol 271:473–492
Stanton GB, Goldberg ME, Bruce CJ (1988b) Frontal eye field efferents in the macaque monkey. II. Topography of terminal fields in idbrain and pons. J Comp Neurol 271:493–506
Van Essen DC (1979) Visual areas of the mammalian cerebral cortex. Annu Rev Neurosci 2:227–263
Wang D, Arbib M (1990) Complex temporal sequence learning based on short-term memory. Proc IEEE 78:1536–1543
Weitzenfield A (1991) NSL neural simulation language version 2. 1. (Technical Report 91-05). Center for Neural Engineering, University of Southern California
Yeterian EH, Pandya D (1991) Prefrontostriatal connections in relation to cortical architectonic organization in rhesus monkeys. J Comp Neurol 312:43–67
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Dominey, P.F. Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning. Biol. Cybern. 73, 265–274 (1995). https://doi.org/10.1007/BF00201428
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00201428