Skip to main content
Springer Nature Link
Log in

Generalization and Interference in Human Motor Control

  • Chapter
  • First Online:
Computational and Robotic Models of the Hierarchical Organization of Behavior

Abstract

Humans (and robots) interacting with the environment have to deal with a continuous stream of sensory inputs in an incremental fashion. Such systems face two fundamental issues: (1) they must acquire new skills in a cumulative fashion, that is exploiting previous knowledge to learn new behaviors, and (2) they must avoid the so-called catastrophic interference, where learning new knowledge destroys existing memories. Here, we analyze the problem from the perspective of biological motor control. We first review experimental results of consolidation of procedural memories and the factors affecting it. Then the problem of generalization and interference is examined together with some interpretations in terms of computational models. Finally, we present some possible approaches to the issue of learning multiple tasks while avoiding catastrophic interference in bio-inspired learning architectures.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The policies and the outcomes may be complex. For example, the outcomes may actually be sequences of elementary observations, while the policies may be made up of sequences of elementary actions.

  2. 2.

    A set is countable if every element of the set can be associated with a natural number. An example of countable set is the set of natural numbers \(\mathbb{N} =\{ 0, 1, 2, 3,\ldots \}\). The set \(\mathbb{R}\) of real numbers is an example of uncountable set.

References

  • Atkeson, C., & Santamaria, J. (1997). A comparison of direct and model-based reinforcement learning. In Proceedings., 1997 IEEE international conference on robotics and automation, 1997 (vol. 4, pp. 3557–3564). New York: IEEE.

    Google Scholar 

  • Barto, A., & Mahadevan, S. (2003). Recent advances in hierarchical reinforcement learning. Discrete Event Dynamic Systems, 13(4), 341–379.

    Article  MathSciNet  Google Scholar 

  • Barto, A., Singh, S., Chentanez, N. (2004). Intrinsically motivated learning of hierarchical collections of skills. In Proceedings of the 3rd international conference on development and learning. ICDL. (pp. 112–119).

    Google Scholar 

  • Battig, W. (1972). Intratask interference as a source of facilitation in transfer and retention. In R. F. Thompson & J. F. Voss (Eds.), Topics on Learning and Performance. New York: Academic Press. (pp. 131–159).

    Google Scholar 

  • Berniker, M., & Kording, K. (2008). Estimating the sources of motor errors for adaptation and generalization. Nature Neuroscience, 11(12), 1454–1461.

    Article  Google Scholar 

  • Bertsekas, D. P., & Tsitsiklis, J. N. (1996). Neuro-dynamic programming. Belmont: Athena Scientific.

    MATH  Google Scholar 

  • Bizzi, E., Accornero, N., Chapple, W., Hogan, N. (1984). Posture control and trajectory formation during arm movement. The Journal of Neuroscience, 4(11), 2738–2744.

    Google Scholar 

  • Braun, D., Aertsen, A., Wolpert, D., Mehring, C. (2009). Motor task variation induces structural learning. Current Biology, 19(4), 352–357.

    Article  Google Scholar 

  • Braun, D., Waldert, S., Aertsen, A., Wolpert, D., Mehring, C., Gribble, P. (2010). Structure learning in a sensorimotor association task. PLoS ONE, 5(1), e8973.

    Article  Google Scholar 

  • Caithness, G., Osu, R., Bays, P., Chase, H., Klassen, J., Kawato, M., Wolpert, D., Flanagan, J. (2004). Failure to consolidate the consolidation theory of learning for sensorimotor adaptation tasks. Journal of Neuroscience, 24(40), 8662–8671.

    Article  Google Scholar 

  • Cheng, G., Hyon, S., Morimoto, J., Ude, A., Hale, J., Colvin, G., Scroggin, W., and Jacobsen, S. (2007). CB: a humanoid research platform for exploring neuroscience. Advanced Robotics, 21(10), 1097–1114.

    Article  Google Scholar 

  • Corkin, S. (1968). Acquisition of motor skill after bilateral medial temporal-lobe excision. Neuropsychologia, 6(3), 255–265.

    Article  Google Scholar 

  • Criscimagna-Hemminger, S., & Shadmehr, R. (2008). Consolidation patterns of human motor memory. Journal of Neuroscience, 28(39), 9610–9618.

    Article  Google Scholar 

  • d’Avella, A., Saltiel, P., Bizzi, E. (2003). Combinations of muscle synergies in the construction of a natural motor behavior. Nature Neuroscience, 6(3), 300.

    Google Scholar 

  • Dayan, P., & Balleine, B. (2002). Reward, motivation, and reinforcement learning. Neuron, 36(2), 285–298.

    Article  Google Scholar 

  • DeGroot, M. H. (1970). Optimal statistical decisions. New York: Wiley.

    MATH  Google Scholar 

  • Dimitrakakis, C. (2012). Sparse reward processes. Technical Report arXiv:1201.2555v1, EPFL.

    Google Scholar 

  • Doya, K., Samejima, K., Katagiri, K., Kawato, M. (2002). Multiple model-based reinforcement learning. Neural computation, 14(6), 1347–1369.

    Article  MATH  Google Scholar 

  • Emadi Andani, M., Bahrami, F., Jabehdar Maralani, P., Ijspeert, A. (2009). MODEM: a multi-agent hierarchical structure to model the human motor control system. Biological Cybernetics, 101(5), 361–377.

    Article  MathSciNet  Google Scholar 

  • Eysenck, H. (1965). A three-factor theory of reminiscence. British Journal of Psychology, 56(163–181), 50–130.

    Google Scholar 

  • Feldman, A. (1966). Functional tuning of the nervous system with control of movement or maintenance of a steady posture. ii. controllable parameters of the muscle. Biophysics, 11(3), 565–578.

    Google Scholar 

  • Ferguson, K., & Mahadevan, S. (2006). Proto-transfer learning in markov decision processes using spectral methods. In Computer Science Department Faculty Publication Series (p. 151). Amherst: University of Massachusetts.

    Google Scholar 

  • French, R. (1997). Pseudo-recurrent Connectionist Networks: An Approach to the “Sensitivity-Stability” Dilemma. Connection Science, 9(4), 353–380.

    Article  Google Scholar 

  • French, R. (1999). Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences, 3(4), 128–135.

    Article  MathSciNet  Google Scholar 

  • Gomi, H., & Kawato, M. (1993). Recognition of manipulated objects by motor learning with modular architecture networks. Neural Networks, 6(4), 485–497.

    Article  Google Scholar 

  • Gomi, H., & Kawato, M. (1996). Equilibrium-point control hypothesis examined by measured arm stiffness during multijoint movement. Science, 272, 117–120.

    Article  Google Scholar 

  • Haruno, M., Wolpert, D., Kawato, M. (2001). Mosaic model for sensorimotor learning and control. Neural Computation, 13(10), 2201–2220.

    Article  MATH  Google Scholar 

  • Haruno, M., Wolpert, D., Kawato, M. (2003). Hierarchical mosaic for movement generation. In International Congress Series (vol. 1250, pp. 575–590). Amsterdam: Elsevier.

    Google Scholar 

  • Hinton, G., & Plaut, D. (1987). Using fast weights to deblur old memories. In Program of the ninth annual conference of the cognitive science society (pp. 177–186). London: Lawrence Erlbaum.

    Google Scholar 

  • Hogan, N. (1984). An organizing principle for a class of voluntary movements. The Journal of Neuroscience, 4(11), 2745–2754.

    Google Scholar 

  • Huang, V., & Shadmehr, R. (2009). Persistence of motor memories reflects statistics of the learning event. Journal of Neurophysiology, 102(2), 931–940.

    Article  Google Scholar 

  • Jacobs, R., Jordan, M., Nowlan, S., Hinton, G. (1991). Adaptive mixtures of local experts. Neural Computation, 3(1), 79–87.

    Article  Google Scholar 

  • Kantak, S., Sullivan, K., Fisher, B., Knowlton, B., Winstein, C. (2010). Neural substrates of motor memory consolidation depend on practice structure. Nature Neuroscience, 13(8), 923–925.

    Article  Google Scholar 

  • Kawato, M. (1999). Internal models for motor control and trajectory planning. Current Opinion in Neurobiology, 9(6), 718–727.

    Article  Google Scholar 

  • Keisler, A., & Shadmehr, R. (2010). A shared resource between declarative memory and motor memory. The Journal of Neuroscience, 30(44), 14817–14823.

    Article  Google Scholar 

  • Kording, K., Tenenbaum, J., Shadmehr, R. (2007). The dynamics of memory as a consequence of optimal adaptation to a changing body. Nature Neuroscience, 10(6), 779–786.

    Article  Google Scholar 

  • Krakauer, J., Ghez, C., Ghilardi, M. (2005). Adaptation to visuomotor transformations: consolidation, interference, and forgetting. Journal of Neuroscience, 25(2), 473–478.

    Article  Google Scholar 

  • Krakauer, J., Mazzoni, P., Ghazizadeh, A., Ravindran, R., Shadmehr, R. (2006). Generalization of motor learning depends on the history of prior action. PLoS Biology, 4(10), e316.

    Article  Google Scholar 

  • Lindley, D. V. (1956). On a measure of the information provided by an experiment. Annals of Mathematical Statistics, 27(4), 986–105.

    Article  MathSciNet  MATH  Google Scholar 

  • Lonini, L., Dipietro, L., Zollo, L., Guglielmelli, E., Krebs, H. (2009). An internal model for acquisition and retention of motor learning during arm reaching. Neural Computation, 21(7), 2009–2027.

    Article  MATH  Google Scholar 

  • Mahadevan, S., & Maggioni, M. (2007). Proto-value functions: a Laplacian framework for learning representation and control in Markov decision processes. Journal of Machine Learning Research, 8, 2169–2231.

    MathSciNet  MATH  Google Scholar 

  • Mandelblat-Cerf, Y., Novick, I., Paz, R., Link, Y., Freeman, S., Vaadia, E. (2011). The neuronal basis of long-term sensorimotor learning. The Journal of Neuroscience, 31(1), 300–313.

    Article  Google Scholar 

  • Medina, J., Garcia, K., Mauk, M. (2001). A mechanism for savings in the cerebellum. Journal of Neuroscience, 21(11), 4081–4089.

    Google Scholar 

  • Mitchell, C., Nash, S., Hall, G. (2008). The intermixed-blocked effect in human perceptual learning is not the consequence of trial spacing. Journal Of Experimental Psychology, Learning, Memory and Cognition, 34(1), 237.

    Google Scholar 

  • Mussa-Ivaldi, F., Giszter, S., Bizzi, E. (1994). Linear combinations of primitives in vertebrate motor control. Proceedings of the National Academy of Sciences, 91(16), 7534.

    Article  Google Scholar 

  • Norman, K., & O’Reilly, R. (2003). Modeling hippocampal and neocortical contributions to recognition memory: A complementary-learning-systems approach. Psychological Review, 110(4), 611–646.

    Article  Google Scholar 

  • Osu, R., Hirai, S., Yoshioka, T., Kawato, M. (2004). Random presentation enables subjects to adapt to two opposing forces on the hand. Nature Neuroscience, 7(2), 111–112.

    Article  Google Scholar 

  • Overduin, S., Richardson, A., Lane, C., Bizzi, E., Press, D. (2006). Intermittent practice facilitates stable motor memories. Journal of Neuroscience, 26(46), 1188–1892.

    Article  Google Scholar 

  • Pekny, S., Criscimagna-Hemminger, S., Shadmehr, R. (2011). Protection and expression of human motor memories. The Journal of Neuroscience, 31(39), 13829–13839.

    Article  Google Scholar 

  • Poggio, T., & Bizzi, E. (2004). Generalization in vision and motor control. Nature, 431(7010), 768–774.

    Article  Google Scholar 

  • Puterman, M. (1994). Markov decision processes: discrete stochastic dynamic programming. New York: Wiley.

    Book  MATH  Google Scholar 

  • Robertson, E. (2007). The serial reaction time task: implicit motor skill learning? Journal of Neuroscience, 27(38), 10073.

    Article  Google Scholar 

  • Robertson, E., Pascual-Leone, A., Miall, R. (2004a). Current concepts in procedural consolidation. Nature Reviews Neuroscience, 5(7), 576–582.

    Article  Google Scholar 

  • Robertson, E., Pascual-Leone, A., Press, D. (2004b). Awareness modifies the skill-learning benefits of sleep. Current Biology, 14(3), 208–212.

    Google Scholar 

  • Rucci, M., Edelman, G., Wray, J. (1999). Adaptation of orienting behavior: from the barn owl to a robotic system. IEEE Transactions on Robotics and Automation, 15(1), 96–110.

    Article  Google Scholar 

  • Schmidhuber, J. (1991). A possibility for implementing curiosity and boredom in model-building neural controllers. In From animals to animats: proceedings of the first international conference on simulation of adaptive behavior (SAB90). Citeseer.

    Google Scholar 

  • Scoville, W., & Milner, B. (1957). Loss of recent memory after bilateral hippocampal lesions. Journal of Neurology, Neurosurgery & Psychiatry, 20(1), 11–21.

    Article  Google Scholar 

  • Shadmehr, R., Smith, M., Krakauer, J. (2010). Error correction, sensory prediction, and adaptation in motor control. Annual Review of Neuroscience, 33, 89–108.

    Article  Google Scholar 

  • Shea, J., & Morgan, R. (1979). Contextual interference effects on the acquisition, retention, and transfer of a motor skill. Journal of Experimental Psychology: Human Learning and Memory, 5(2), 179–187.

    Article  Google Scholar 

  • Singh, S., Lewis, R., Barto, A., Sorg, J. (2010). Intrinsically motivated reinforcement learning: an evolutionary perspective. IEEE Transactions on Autonomous Mental Development, 2(2), 70–82.

    Article  Google Scholar 

  • Smith, M., Ghazizadeh, A., Shadmehr, R. (2006). Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biology, 4(6), e179.

    Article  Google Scholar 

  • Spencer, R., Sunm, M., Ivry, R. (2006). Sleep-dependent consolidation of contextual learning. Current Biology, 16(10), 1001–1005.

    Article  Google Scholar 

  • Sutton, R., Precup, D., Singh, S. (1999). Between mdps and semi-mdps: a framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112(1), 181–211.

    Article  MathSciNet  MATH  Google Scholar 

  • Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: an introduction. Cambridge: Cambridge University Press.

    Google Scholar 

  • Taylor, M., & Stone, P. (2009). Transfer learning for reinforcement learning domains: a survey. The Journal of Machine Learning Research, 10, 1633–1685.

    MathSciNet  MATH  Google Scholar 

  • Thoroughman, K., & Shadmehr, R. (2000). Learning of action through adaptive combination of motor primitives. Nature, 407(6805), 742.

    Article  Google Scholar 

  • Valero-Cuevas, F., Venkadesan, M., Todorov, E. (2009). Structured variability of muscle activations supports the minimal intervention principle of motor control. Journal of Neurophysiology, 102(1), 59.

    Article  Google Scholar 

  • Vijayakumar, S., D’souza, A., Schaal, S. (2005). Incremental online learning in high dimensions. Neural Computation, 17(12), 2602–2634.

    Article  MathSciNet  Google Scholar 

  • Walker, M., Brakefield, T., Hobson, J., Stickgold, R. (2003). Dissociable stages of human memory consolidation and reconsolidation. Nature, 425(6958), 616–620.

    Article  Google Scholar 

  • Wei, K., & Kording, K. (2009). Relevance of error: what drives motor adaptation? Journal of Neurophysiology, 101(2), 655.

    Article  Google Scholar 

  • Westermann, G., & Mareschal, D. (2008). A dual-memory model of categorization in infancy. In From associations to rules: connectionist models of behavior and cognition: proceedings of the tenth Neural Computation and Psychology Workshop, Dijon, France, 12–14 April 2007 (vol. 17, p. 127). Singapore: World Scientific Pub Co Inc.

    Google Scholar 

  • Wolpert, D., & Kawato, M. (1998). Multiple paired forward and inverse models for motor control. Neural Networks, 11(7–8), 1317–1329.

    Article  Google Scholar 

  • Yamashita, Y., & Tani, J. (2008). Emergence of functional hierarchy in a multiple timescale neural network model: a humanoid robot experiment. PLoS Computational Biology, 4(11), e1000220.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Frankfurt Institute for Advanced Studies, Goethe University Frankfurt, Ruth-Moufang Str. 1, 60438, Frankfurt am Main, Germany

    Luca Lonini

  2. Chalmers University of Technology Sweden, SE-412 96, Gothenburg, Sweden

    Christos Dimitrakakis

  3. Institute of Psychology Organization: Technical University Darmstadt, Karolinenplatz 5, 64289, Darmstadt, Germany

    Constantin Rothkopf

  4. Frankfurt Institute for Advanced Studies, Goethe University Frankfurt, Frankfurt, Germany

    Jochen Triesch

Authors
  1. Luca Lonini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christos Dimitrakakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Constantin Rothkopf
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jochen Triesch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luca Lonini .

Editor information

Editors and Affiliations

  1. Consiglio Nazionale delle Ricerche, Istituto di Scienze e Tecnologie della Cognizione, Rome, Italy

    Gianluca Baldassarre

  2. Consiglio Nazionale delle Ricerche, Istituto di Scienze e Tecnologie della Cognizione, Rome, Italy

    Marco Mirolli

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Lonini, L., Dimitrakakis, C., Rothkopf, C., Triesch, J. (2013). Generalization and Interference in Human Motor Control. In: Baldassarre, G., Mirolli, M. (eds) Computational and Robotic Models of the Hierarchical Organization of Behavior. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39875-9_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-39875-9_8

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-39874-2

  • Online ISBN: 978-3-642-39875-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation