Skip to main content

Advertisement

SpringerLink
Log in

On-line learning and modulation of periodic movements with nonlinear dynamical systems

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

The paper presents a two-layered system for (1) learning and encoding a periodic signal without any knowledge on its frequency and waveform, and (2) modulating the learned periodic trajectory in response to external events. The system is used to learn periodic tasks on a humanoid HOAP-2 robot. The first layer of the system is a dynamical system responsible for extracting the fundamental frequency of the input signal, based on adaptive frequency oscillators. The second layer is a dynamical system responsible for learning of the waveform based on a built-in learning algorithm. By combining the two dynamical systems into one system we can rapidly teach new trajectories to robots without any knowledge of the frequency of the demonstration signal. The system extracts and learns only one period of the demonstration signal. Furthermore, the trajectories are robust to perturbations and can be modulated to cope with a dynamic environment. The system is computationally inexpensive, works on-line for any periodic signal, requires no additional signal processing to determine the frequency of the input signal and can be applied in parallel to multiple dimensions. Additionally, it can adapt to changes in frequency and shape, e.g. to non-stationary signals, such as hand-generated signals and human demonstrations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Buchli, J., & Ijspeert, A. (2004). A simple, adaptive locomotion toy-system. In S. Schaal, A. Ijspeert, A. Billard, S. Vijayakumar, J. Hallam, & J. Meyer (Eds.), From animals to animats 8. Proceedings of the eighth international conference on the simulation of adaptive behavior (SAB’04) (pp. 153–162). Cambridge: MIT.

    Google Scholar 

  • Buchli, J., Righetti, L., & Ijspeert, A. (2006). Engineering entrainment and adaptation in limit cycle systems—from biological inspiration to applications in robotics. Biological Cybernetics, 95(6), 645–664.

    Article  MATH  MathSciNet  Google Scholar 

  • Bullock, D., & Grossberg, S. (1989). VITE and FLETE: neural modules for trajectory formation and postural control. In W. Hersberger (Ed.), Volitional control (pp. 253–297). Amsterdam: Elsevier.

    Chapter  Google Scholar 

  • Calinon, S., Guenter, F., & Billard, A. (2007). On learning, representing and generalizing a task in a humanoid robot. IEEE Transactions on Systems, Man and Cybernetics, Part B, Special issue on robot learning by observation, demonstration and imitation, 37(2), 286–298.

    Google Scholar 

  • Crespi, A., Badertscher, A., & Guignard, A. (2005). AmphiBot I: An amphibious snake-like robot. Robotics and Autonomous Systems, 50(4), 163–175.

    Article  Google Scholar 

  • Drumwright, E., Jenkins, O. C., & Mataric, M. (2004). Exemplar-based primitives for humanoid movement classification and control. In Proceedings of the 2004 IEEE international conference on robotics and automation (ICRA 2004), pp. 140–145.

  • Fod, A., Mataric, M., & Jenkins, O. C. (2002). Automated derivation of primitives for movement classification. Autonomous Robots, 12(1), 39–54.

    Article  MATH  Google Scholar 

  • Gams, A., Zlajpah, L., & Lenarcic, J. (2007). Imitating human acceleration of a gyroscopic device. Robotica, 25(4), 501–509.

    Article  Google Scholar 

  • Grimes, D., Rashid, D., & Rao, R. (2006). Learning nonparametric models for probabilistic imitation. In NIPS, pp. 521–528.

  • Guenter, F., Hersch, M., Calinon, S., & Billard, A. (2007). Reinforcement learning for imitating constrained reaching movements. RSJ Advanced Robotics, Special Issue on Imitative Robots, 21(13), 1521–1544.

    Google Scholar 

  • Harris, C. M., & Wolpert, D. M. (1998). Signal-dependent noise determines motor planning. Nature, 394(6695), 780–784. doi:10.1038/29528.

    Article  Google Scholar 

  • Hersch, M., Guenter, F., Calinon, S., & Billard, A. (2008). Dynamical system modulation for robot learning via kinesthetic demonstrations. IEEE Transactions on Robotics.

  • Ijspeert, A., Hallam, J., & Willshaw, D. (1999). Evolving swimming controllers for a simulated lamprey with inspiration from neurobiology. Adaptive Behavior, 7(2), 151–172.

    Article  Google Scholar 

  • Ijspeert, A., Nakanishi, J. & Schaal, S. (2002a). Learning attractor landscapes for learning motor primitives. In Advances in neural information processing systems 15 (NIPS2002).

  • Ijspeert, A., Nakanishi, J., & Schaal, S. (2002b). Learning rhythmic movements by demonstration using nonlinear oscillators. In Proceedings of the IEEE/RSJ Int. Conference on Intelligent Robots and Systems (IROS2002) (pp. 958–963).

  • Ijspeert, A., Nakanishi, J., & Schaal, S. (2002c). Movement imitation with nonlinear dynamical systems in humanoid robots. In IEEE International Conference on Robotics and Automation (ICRA2002) (pp. 1398–1403).

  • Ijspeert, A. J. (2008). Central pattern generators for locomotion control in animals and robots: a review. Neural Networks, 21(4), 642–653.

    Article  Google Scholar 

  • Inamura, T., Toshima, I., Tanie, H., & Nakamura, Y. (2004). Embodied symbol emergence based on mimesis theory. International Journal of Robotic Research, 23(4–5), 363–377.

    Article  Google Scholar 

  • Kawamura, S., & Fukao, N. (1994). Interpolation for input torque patterns obtained through learning control. In Proceedings of the third international conference on automation, robotics and computer vision (ICARCV’94).

  • Kawato, M. (1996). Trajectory formation in arm movements: minimization principles and procedures. In H. Zelaznik (Ed.), Advance in motor learning and control (pp. 225–259). Champaign: Human Kinetics Publisher.

    Google Scholar 

  • Khatib, O. (1986). Real-time obstacle avoidance for manipulators and mobile robots. International Journal of Robotic Research, 5(1), 90–98.

    Article  MathSciNet  Google Scholar 

  • Li, P., & Horowitz, R. (1999). Passive velocity field control of mechanical manipulators. IEEE Transactions on Robotics and Automation, 15(4), 751–763.

    Article  Google Scholar 

  • Ljung, L., & Söderström, T. (1986). Theory and Practice of Recursive Identification. Cambridge: MIT.

    Google Scholar 

  • Maass, W., Natschläger, T., & Markram, H. (2002). Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11), 2531–2560.

    Article  MATH  Google Scholar 

  • Mataric, M. (1998). Behavior-based robotics as a tool for synthesis of artificial behavior and analysis of natural behavior. Trends in Cognitive Science, 2(3), 82–87.

    Article  Google Scholar 

  • Mezger, J. W. I., & Giese, M. (2005). Trajectory synthesis by hierarchical spatio-temporal correspondence: comparison of different methods. In APGV ’05: Proceedings of the 2nd symposium on Applied perception in graphics and visualization (pp. 25–32). New York: ACM.

    Chapter  Google Scholar 

  • Miyamoto, H., Schaal, S., Gandolfo, F., Koike, Y., Osu, R., Nakano, E., Wada, Y., & Kawato, M. (1996). A Kendama learning robot based on bi-directional theory. Neural Networks, 9, 1281–1302.

    Article  Google Scholar 

  • Mussa-Ivaldi, F. (1997). Nonlinear force fields: a distributed system of control primitives for representing and learning movements. In IEEE International Symposium on Computational Intelligence in Robotics and Automation (pp. 84–90). Los Alamitos: IEEE, Computer Society.

    Google Scholar 

  • Nishii, J. (1998). A learning model for oscillatory networks. Neural Networks, 11(2), 249–257.

    Article  MathSciNet  Google Scholar 

  • Okada, M., Tatani, K., & Nakamura, Y. (2002). Polynomial design of the nonlinear dynamics for the brain-like information processing of whole body motion. In Proceedings of ICRA 2002 (Vol. 2, pp. 1410–1415). IEEE.

  • Paine, R. W., & Tani, J. (2004). Motor primitive and sequence self-organization in a hierarchical recurrent neural network. Neural Networks, 17, 1291–1309.

    Article  Google Scholar 

  • Righetti, L., & Ijspeert, A. (2006). Programmable central pattern generators: an application to biped locomotion control. In Proceedings of the 2006 IEEE international conference on robotics and automation (pp. 1585–1590).

  • Righetti, L., Buchli, J., & Ijspeert, A. (2006). Dynamic Hebbian learning in adaptive frequency oscillators. Physica D, 216(2), 269–281.

    Article  MATH  MathSciNet  Google Scholar 

  • Rohrer, B., & Hogan, N. (2003). Avoiding spurious submovement decompositions: a globally optimal algorithm. Biological Cybernetics, 89(3), 190–199.

    Article  MATH  Google Scholar 

  • Schaal, S. (1999). Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3, 233–242.

    Article  Google Scholar 

  • Schaal, S., & Atkeson, C. (1998). Constructive incremental learning from only local information. Neural Computation, 10, 2047–2084.

    Article  Google Scholar 

  • Schaal, S., Mohajerian, P., & Ijspeert, A. (2007). Dynamics systems vs. optimal control a unifying view. Progress in Brain Research, 165, 425–445.

    Article  Google Scholar 

  • Simard, P., & LeCun, Y. (1991). Reverse TDNN: an architecture for trajectory generation. In NIPS (pp. 579–588).

  • Tani, J., & Ito, M. (2003). Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment. IEEE Transactions on Systems, Man, and Cybernetics, 33(4), 481–488.

    Article  Google Scholar 

  • Tsuji, T., Tanaka, Y., Morasso, P., Sanguineti, V., & Kaneko, M. (2002). Bio-mimetic trajectory generation of robots via artificial potential field with time base generator. Systems, Man and Cybernetics, Part C, IEEE Transactions on, 32(4), 426–439.

    Article  Google Scholar 

  • Ude, A., Riley, M., Nemec, B., Kos, A., Asfour, T., & Cheng, G. (2007). Synthesizing goal-directed actions from a library of example movements. In IEEE-RAS/RSJ international conference on humanoid robots (Humanoids 2007).

  • Zegers, P., & Sundareshan, M. (2003). Trajectory generation and modulation using dynamic neural networks. IEEE Transactions on Neural Networks, 14(3), 520–533.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. “Jožef Stefan” Institute, Jamova cesta 39, 1000, Ljubljana, Slovenia

    Andrej Gams & Jadran Lenarčič

  2. School of Computer and Communication Sciences, EPFL—École Polytechnique Fédérale de Lausanne, Station 14, 1015, Lausanne, Switzerland

    Auke J. Ijspeert

  3. Computational Learning and Motor Control Laboratory, University of Southern California, Los Angeles, CA, 90089-2520, USA

    Stefan Schaal

Authors
  1. Andrej Gams
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Auke J. Ijspeert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefan Schaal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jadran Lenarčič
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrej Gams.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gams, A., Ijspeert, A.J., Schaal, S. et al. On-line learning and modulation of periodic movements with nonlinear dynamical systems. Auton Robot 27, 3–23 (2009). https://doi.org/10.1007/s10514-009-9118-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-009-9118-y

Keywords

Search

Navigation