Skip to main content
SpringerLink
Log in

The SURE_REACH Model for Motor Learning and Control of a Redundant Arm: From Modeling Human Behavior to Applications in Robotics

  • Chapter
Book cover From Motor Learning to Interaction Learning in Robots

Part of the book series: Studies in Computational Intelligence ((SCI,volume 264))

Abstract

The recently introduced neural network SURE_REACH (sensorimotor unsupervised redundancy resolving control architecture) models motor cortical learning and control of human reaching movements. The model learns redundant, internal body models that are highly suitable to flexibly invoke effective motor commands. The encoded redundancy is used to adapt behavior flexible to situational constraints without the need for further learning. These adaptations to specific tasks or situations are realized by a neurally generated movement plan that adheres to various end-state or trajectory-related constraints. The movement plan can be implemented by proprioceptive or visual closed-loop control. This chapter briefly reviews the literature on computational models of motor learning and control and gives a description of SURE_REACH and its neural network implementation. Furthermore, we relate the model to human motor learning and performance and discuss its neural foundations. Finally, we apply the model to the control of a dynamic robot platform. In sum, SURE_REACH grounds highly flexible task-dependent behavior on a neural network framework for unsupervised learning. It accounts for the neural processes that underlie fundamental aspects of human behavior and is well applicable to the control of robots.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adams, J.A.: A closed-loop theory of motor learning. J. Mot. Behav. 3(2), 111–149 (1971)

    Google Scholar 

  2. Aflalo, T.N., Graziano, M.S.A.: Partial tuning of motor cortex neurons to final posture in a free-moving paradigm. Proc. Natl. Acad. of Sci. 8, 2909–2914 (2006)

    Article  Google Scholar 

  3. Baraduc, P., Guigon, E., Burnod, Y.: Where does the population vector of motor cortical cells point during reaching movements? In: Kearns, M., Solla, S., Cohn, D. (eds.) Advances in Neural Information Processing Systems, vol. 11, pp. 83–89. MIT Press, Cambridge (1999)

    Google Scholar 

  4. Barto, A.G., Fagg, A.H., Sitkoff, N., Houk, J.C.: A cerebellar model of timing and prediction in the control of reaching. Neural Comp. 11, 565–594 (1999)

    Article  Google Scholar 

  5. Bastian, A., Schöner, G., Riehle, A.: Preshaping and continuous evolution of motor cortical representations during movement preparation. Eur. J. Neurosci. 18, 2047–2058 (2003)

    Article  Google Scholar 

  6. Bernstein, N.A.: The co-ordination and regulation of movements. Pergamon Press, Oxford (1967)

    Google Scholar 

  7. Berthier, N.E., Rosenstein, M.T., Barto, A.G.: Approximate optimal control as a model for motor learning. Psychol. Rev. 112(2), 329–346 (2005)

    Article  Google Scholar 

  8. Bowers, J.S.: On the biological plausibility of grandmother cells: Implications for neural network theories in psychology and neuroscience. Psychol. Rev. 116(1), 220–251 (2009)

    Article  Google Scholar 

  9. Bullock, D., Grossberg, S., Guenther, F.H.: A self-organizing neural model of motor equivalent reaching and tool use by a multijoint arm. J. Cogn. Neurosci. 5(4), 408–435 (1993)

    Article  Google Scholar 

  10. Butz, M.V., Herbort, O., Hoffmann, J.: Exploiting redundancy for flexible behavior: Unsupervised learning in a modular sensorimotor control architecture. Psychol. Rev. 114(4), 1015–1046 (2007)

    Article  Google Scholar 

  11. Butz, M.V., Herbort, O., Pezzulo, G.: Anticipatory, goal-directed behavior. In: Pezzulo, G., Butz, M.V., Castelfranchi, C., Falcone, R. (eds.) The Challenge of Anticipation. LNCS (LNAI), vol. 5225, pp. 85–113. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  12. Cisek, P.: Integrated neural processes for defining potential actions and deciding between them: A computational model. J. Neurosci. 26(38), 9761–9770 (2006)

    Article  Google Scholar 

  13. Cisek, P., Kalaska, J.F.: Neural correlates of reaching decisions in dorsal premotor cortex: Specification of multiple direction choices and final selection of action. Neuron 45(5), 801–814 (2005)

    Article  Google Scholar 

  14. Cruse, H., Steinkühler, U.: Solution of the direct and inverse kinematic problems by a common algorithm based on the mean of multiple computations. Biol. Cybern. 69, 341–351 (1993)

    Article  Google Scholar 

  15. Cruse, H., Steinkühler, U., Burkamp, C.: MMC - a recurrent neural network which can be used as manipulable body model. In: Pfeifer, R., Blumberg, B., Meyer, J.A., Wilson, S. (eds.) From Animals to Animats 5: The Fifth International Conference on the Simulation of Adaptive Behavior, pp. 381–389. MIT Press, Cambridge (1998)

    Google Scholar 

  16. de Freitas, S.M.S.F., Scholz, J.P., Stehman, A.J.: Effect of motor planning on use of motor abundance. Neurosci. Lett. 417(1), 66–71 (2007)

    Article  Google Scholar 

  17. Dean, J., Brüwer, M.: Control of human arm movements in two dimensions: Paths and joint control in avoiding simple linear obstacles. Exp. Brain Res. 97, 497–514 (1994)

    Article  Google Scholar 

  18. Desmurget, M., Grafton, S.: Forward modeling allows feedback control for fast reaching movements. Trends Cogn. Sci. 4(11), 423–431 (2000)

    Article  Google Scholar 

  19. Doya, K.: Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr. Opin. Neurobiol. 10(6), 732–739 (2000)

    Article  Google Scholar 

  20. Elliott, D., Helsen, W.F., Chua, R.: A century later: Woodworth’s two-component model of goal-directed aiming. Psychol. Bull. 127(3), 342–357 (1899)

    Article  Google Scholar 

  21. Elsner, B., Hommel, B.: Effect anticipations and action control. J. Exp. Psychol. 27(1), 229–240 (2001)

    Google Scholar 

  22. Fischer, M.H., Rosenbaum, D.A., Vaughan, J.: Speed and sequential effects in reaching. J. Exp. Psychol.: Hum. Percept. Perform. 23(2), 404–428 (1997)

    Article  Google Scholar 

  23. Flash, T., Hogan, N.: The coordination of arm movements: An experimentally confirmed mathematical model. J. Neurosci. 5(7), 1688–1703 (1985)

    Google Scholar 

  24. Georgopoulos, A.P.: Current issues in directional motor control. Trends Neurosci. 18(11), 506–510 (1995)

    Article  Google Scholar 

  25. Hallett, M.A., Pascual-Leone, A., Topka, H.: The acquisition of motor behavior in vertebrates. In: Bloedel, J.R., Ebner, T.J., Wise, S.P. (eds.) Adaptation and skill learning: Evidence for different neural substrates, pp. 289–301. MIT Press, Cambridge (1996)

    Google Scholar 

  26. Haruno, M., Wolpert, D.M., Kawato, M.: Hierarchical mosaic for movement generation. In: Ono, T., Matsumoto, G., Llinas, R., Berthoz, A., Norgren, R., Nishijo, H., Tamura, R. (eds.) Excepta Medica International Congress Series, vol. 1250 (2003)

    Google Scholar 

  27. Herbart, J.F.: Psychologie als Wissenschaft neu gegründet auf Erfahrung, Metaphysik und Mathematik. In: Zweiter analytischer Teil [Psychology as a Science newly founded on Experience, Metaphysics and Mathematics: Second, Analytical Part]. August Wilhelm Unzer., Königsberg (1825)

    Google Scholar 

  28. Herbort, O., Butz, M.V.: Encoding complete body models enables task dependent optimal control. Proc. Int. Jt. Conf. Neural Netw. 20, 1639–1644 (2007)

    Article  Google Scholar 

  29. Herbort, O., Butz, M.V.: Anticipatory planning of sequential hand and finger movements. J. Mot. Behav. (in press)

    Google Scholar 

  30. Herbort, O., Butz, M.V., Hoffmann, J.: Multimodal goal representations and feedback in hierarchical motor control. In: Proc. Int. Conf. Cogn. Syst. (2008)

    Google Scholar 

  31. Herbort, O., Ognibene, D., Butz, M.V., Baldassarre, G.: Learning to select targets within targets in reaching tasks. In: Proc. 6th Int. IEEE Conf. Dev. Learn., vol. 6, pp. 7–12 (2007)

    Google Scholar 

  32. Hikosaka, O., Nakamura, K., Sakai, K., Nakahara, H.: Central mechanisms of motor skill learning. Curr. Opin. Neurobiol. 12, 217–222 (2002)

    Article  Google Scholar 

  33. Hoffmann, J.: Vorhersage und Erkenntnis: Die Funktion von Antizipationen in der menschlichen Verhaltenssteuerung und Wahrnehmung [Anticipation and cognition: The function of anticipations in human behavioral control and perception]. Hogrefe, Göttingen (1993)

    Google Scholar 

  34. Hoffmann, J.: Anticipatory behavior control. In: Butz, M.V., Sigaud, O., Gérard, P. (eds.) Anticipatory Behavior in Adaptive Learning Systems. LNCS (LNAI), vol. 2684, pp. 44–65. Springer, Heidelberg (2003)

    Google Scholar 

  35. Hoffmann, J., Butz, M.V., Herbort, O., Kiesel, A., Lenhard, A.: Spekulationen zur Struktur ideo-motorischer Beziehungen [Speculations about the structure of ideomotor relations]. Z. Sportpsychol. 14(3), 95–103 (2007)

    Article  Google Scholar 

  36. Jackson, A., Mavoori, J., Fetz, E.E.: Long-term motor cortex plasticity induced by an electronic neural implant. Nat. 444, 56–60 (2006)

    Article  Google Scholar 

  37. James, W.: The principles of psychology, vol. 1. Holt, New York (1890)

    Google Scholar 

  38. Johnson-Frey, S.H., McCarty, M.E., Keen, R.: Reaching beyond spatial perception: Effects of intended future actions on visually guided prehension. Vis. Cogn. 11(2-3), 371–399 (2004)

    Article  Google Scholar 

  39. Jordan, M.I., Wolpert, D.M.: Computational motor control. In: Gazzaniga (ed.) The Cognitive Neuroscience, pp. 601–620. MIT Press, Cambridge (1999)

    Google Scholar 

  40. Karniel, A., Inbar, G.F.: A model for learning human reaching movements. Biol. Cybern. 77, 173–183 (1997)

    Article  MATH  Google Scholar 

  41. Kawato, M.: Feedback-error-learning neural network for supervised learning. In: Eckmiller, R. (ed.) Advanced neural computers, pp. 365–372. North-Holland, Amsterdam (1990)

    Google Scholar 

  42. Kawato, M.: Internal models for motor control and trajectory planning. Curr. Opin. Neurobiol. 9, 718–727 (1999)

    Article  Google Scholar 

  43. Kawato, M., Furukawa, K., Suzuki, R.: A hierarchical neural-network model for control and learning of voluntary movement. Biol. Cybern. 57, 169–185 (1987)

    Article  MATH  Google Scholar 

  44. Khan, M.A., Franks, I.M., Goodman, D.: The effect of practice on the control of rapid aiming movements: Evidence for an interdependency between programming and feedback processing. Q. J. Exp. Psychol. Section A 51(2), 425–443 (1998)

    Google Scholar 

  45. Klapp, S.T., Erwin, C.I.: Relation between programming time and duration of the response being programmed. J. Exp. Psychol.: Hum. Percept. Perform. 2(4), 591–598 (1976)

    Article  Google Scholar 

  46. Klein Breteler, M.D., Hondzinski, J.M., Flanders, M.: Drawing sequences of segments in 3d: Kinetic influences on arm configuration. J. Neurophysiol. 89, 3253–3263 (2003)

    Article  Google Scholar 

  47. Körding, K.P., Wolpert, D.M.: Bayesian integration in sensorimotor learning. Nat. 427, 244–247 (2004)

    Article  Google Scholar 

  48. Kunde, W., Koch, I., Hoffmann, J.: Anticipated action effects affect the selection, initiation, and execution of actions. Q. J. Exp. Psychol. Section A: Human Exp. Psychol. 57, 87–106 (2004)

    Article  Google Scholar 

  49. Kuperstein, M.: Neural model of adaptive hand-eye coordination for single postures. Sci. 239, 1308–1311 (1988)

    Article  Google Scholar 

  50. Latash, M.L., Scholz, J.P., Schöner, G.: Motor control strategies revealed in the structure of motor variability. Exerc. & Sport Sci. Rev. 30(1), 26–31 (2002)

    Article  Google Scholar 

  51. Lavrysen, A., Helsen, W.F., Tremblay, L., Elliott, D., Adam, J.J., Feys, P., Buekers, M.J.: The control of sequential aiming movements: The influence of practice and manual asymmetries on the one-target advantage. Cortex 39, 307–325 (2003)

    Article  Google Scholar 

  52. Lotze, H.R.: Medicinische Psychologie oder Physiologie der Seele [Medical Psychology or Physiology of the Soul]. Weidmannsche Buchhandlung, Leipzig (1852)

    Google Scholar 

  53. Ludwig, D.A.: Emg changes during the acquisition of a motor skill. Am. J. Phys. Medicine 61(5), 229–243 (1982)

    Google Scholar 

  54. Ma-Wyatt, A., McKee, S.P.: Visual information throughout a reach determines endpoint precision. Exp. Brain Res. 179(1), 55–64 (2007)

    Article  Google Scholar 

  55. Morasso, P., Sanguineti, V., Spada, G.: A computational theory of targeting movements based on force fields and topology representing networks. Neurocomputing 15(3-4), 411–434 (1997)

    Article  Google Scholar 

  56. Munro, H., Plumb, M.S., Wilson, A.D., Williams, J.H.G., Mon-Williams, M.: The effect of distance on reaction time in aiming movements. Exp. Brain Res. 183(2), 249–257 (2007)

    Article  Google Scholar 

  57. Mutsaarts, M., Steenbergen, B., Bekkering, H.: Anticipatory planning deficits and task context effects in hemiparetic cerebral palsy. Exp. Brain Res. 172(2), 151–162 (2006)

    Article  Google Scholar 

  58. Ognibene, D., Mannella, F., Pezzulo, G., Baldassarre, G.: Integrating reinforcement-learning, accumulator models, and motor-primitives to study action selection and reaching in monkeys. In: Fum, D., Del Missier, F., Stocco, A. (eds.) Proc. Seventh International Conference on Cognitive Modeling (ICCM 2006), pp. 214–219. Edizioni Goliardiche, Trieste (2006)

    Google Scholar 

  59. Pedersen, G., Butz, M.V., Herbort, O.: Integrating dynamics into a human behavior model for highly flexible autonomous manipulator control. IEEE Systems, Man & Cybernetics B (submitted)

    Google Scholar 

  60. Peters, J., Schaal, S.: Learning to control in operational space. Int. J. Robot. Res. 27, 197–212 (2008)

    Article  Google Scholar 

  61. Robertson, E.M., Miall, R.C.: Multi-joint limbs permit a flexible response to unpredictable events. Exp. Brain Res. 117, 148–152 (1997)

    Article  Google Scholar 

  62. Rosenbaum, D.A.: Human movement initiation: Specification of arm, direction and extent. J. Exp. Psychol.: Gen. 109, 444–474 (1980)

    Article  Google Scholar 

  63. Rosenbaum, D.A., Engelbrecht, S.E., Bushe, M.M., Loukopoulos, L.D.: A model for reaching control. Acta Psychol. 82(1-3), 237–250 (1993)

    Article  Google Scholar 

  64. Rosenbaum, D.A., Loukopoulos, L.D., Meulenbroek, R.G.J., Vaughan, J., Engelbrecht, S.E.: Planning reaches by evaluating stored postures. Psychol. Rev. 102(1), 28–67 (1995)

    Article  Google Scholar 

  65. Rosenbaum, D.A., Meulenbroek, R.G.J., Vaughan, J., Jansen, C.: Posture-based motion planning: Applications to grasping. Psychol. Rev. 108(4), 709–734 (2001)

    Article  Google Scholar 

  66. Salaun, C., Padois, V., Sigaud, O.: Learning forward models for the operational space control of redundant robots. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 169–192. Springer, Heidelberg (2010)

    Google Scholar 

  67. Schmidt, R.A.: A schema theory of discrete motor skill-learning. Psychol. Rev. 82(4), 229–261 (1975)

    Article  Google Scholar 

  68. Schwartz, A.B., Moran, D.W., Reina, G.A.: Differential representation of perception and action in the frontal cortex. Sci. 303, 380–383 (2004)

    Article  Google Scholar 

  69. Shadmehr, R., Wise, S.P.: The Computational Neurobiology of Reaching and Pointing: A foundation for motor learning. MIT Press, Cambridge (2005)

    Google Scholar 

  70. Short, M.W., Cauraugh, J.H.: Precision hypothesis and the end-state comfort effect. Acta Psychol. 100(3), 243–252 (1999)

    Article  Google Scholar 

  71. Soechting, J.F., Buneo, C.A., Herrmann, U., Flanders, M.: Moving effortlessly in three dimensions: Does Donders’ law apply to arm movement? J. Neurosci. 15, 6271–6280 (1995)

    Google Scholar 

  72. Todorov, E., Jordan, M.I.: Optimal feedback control as a theory of motor coordination. Nat. Neurosci. 5(11), 1226–1235 (2002)

    Article  Google Scholar 

  73. Toussaint, M., Goerick, C.: A bayesian view on motor control and planning. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 227–252. Springer, Heidelberg (2010)

    Google Scholar 

  74. Uno, Y., Kawato, M., Suzuki, R.: Formation and control of optimal trajectory in human multijoint arm movement: Minimum torque-change model. Biol. Cybern. 61(2), 89–101 (1989)

    Article  Google Scholar 

  75. van Sonderen, J.F., Dernier van der Gon, J.J.: Reaction-time-dependent differences in the initial movement direction of fast goal-directed arm movements. Hum. Mov. Sci. 10(6), 713–726 (1991)

    Article  Google Scholar 

  76. von Hofsten, C.: An action perspective on motor development. Trends Cogn. Sci. 8(6), 266–272 (2004)

    Article  Google Scholar 

  77. Webots, C.L.: Commercial mobile robot simulation software, http://www.cyberbotics.com

  78. Weigelt, M., Kunde, W., Prinz, W.: End-state comfort in bimanual object manipulation. Exp. Psychol. 53(2), 143–148 (2006)

    Google Scholar 

  79. Woodworth, R.S.: The Accuracy of Voluntary Movement. New Era, Lancaster (1899)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Universität Würzburg, Röntgenring 11, 97070, Würzburg, Germany

    Oliver Herbort, Martin V. Butz & Gerulf Pedersen

Authors
  1. Oliver Herbort
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin V. Butz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerulf Pedersen
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institut des Systèmes Intelligents et de Robotique (CNRS UMR 7222), Université Pierre et Marie Curie Pyramide, Tour 55 Boîte courrier 173, 4 Place Jussieu, 75252, PARIS cedex 05, France

    Olivier Sigaud

  2. Dept. Schölkopf, Max-Planck Institute for Biological Cybernetics, Spemannstraße 38,Rm 223, 72076, Tübingen, Germany

    Jan Peters

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Herbort, O., Butz, M.V., Pedersen, G. (2010). The SURE_REACH Model for Motor Learning and Control of a Redundant Arm: From Modeling Human Behavior to Applications in Robotics. In: Sigaud, O., Peters, J. (eds) From Motor Learning to Interaction Learning in Robots. Studies in Computational Intelligence, vol 264. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05181-4_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-05181-4_5

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-05180-7

  • Online ISBN: 978-3-642-05181-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation