Abstract
Although motor primitives (MPs) have been studied extensively, much less attention has been devoted to studying their generalization to new situations. To cope with varying conditions, a MP’s policy encoding must support generalization over task parameters to avoid learning separate primitives for each condition. Local and linear parameterized models have been proposed to interpolate over task parameters to provide limited generalization.
In this paper, we present a global parametric motion primitive (GPDMP) which allows generalization beyond local or linear models. Primitives are modeled using a linear basis function model with global non-linear basis functions. The model is constructed from initial non-parametric primitives found using a single human demonstration and subsequent episodes of reinforcement learning to adapt the demonstrated skill to other task parameters. The initial models are then used to optimize the parameters of the global parametric model. Experiments with a ball-in-a-cup task with varying string lengths show that GPDMP allows greatly improved extrapolation compared to earlier local or linear models.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning from demonstration. Robot. Autonom. Syst. 57(5), 469–483 (2009)
Chernova, S., Veloso, M.: Confidence-based policy learning from demonstration using Gaussian mixture models. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, p. 233. ACM (2007)
Sammut, C., Hurst, S., Kedzier, D., Michie, D., et al.: Learning to fly. In: Proceedings of the Ninth International Workshop on Machine Learning, pp. 385–393 (1992)
Hovland, G.E., Sikka, P., McCarragher, B.J.: Skill acquisition from human demonstration using a hidden Markov model. In: IEEE International Conference on Robotics and Automation, Proceedings, vol. 3, pp. 2706–2711. IEEE (1996)
Calinon, S., D’halluin, F., Sauser, E.L., Caldwell, D.G., Billard, A.G.: Learning and reproduction of gestures by imitation. IEEE Robot. Autom. Mag. 17(2), 44–54 (2010)
Carrera, A., Palomeras, N., Hurtós, N., Kormushev, P., Carreras, M.: Learning multiple strategies to perform a valve turning with underwater currents using an I-AUV. In: OCEANS 2015-Genova, pp. 1–8. IEEE (2015)
Da Silva, B., Konidaris, G., Barto, A.: Learning parameterized skills. arXiv preprint arXiv:1206.6398 (2012)
Forte, D., Gams, A., Morimoto, J., Ude, A.: On-line motion synthesis and adaptation using a trajectory database. Robot. Autonom. Syst. 60(10), 1327–1339 (2012)
Ude, A., Gams, A., Asfour, T., Morimoto, J.: Task-specific generalization of discrete and periodic dynamic movement primitives. IEEE Trans. Robot. 26(5), 800–815 (2010)
Matsubara, T., Hyon, S.-H., Morimoto, J.: Learning parametric dynamic movement primitives from multiple demonstrations. Neural Netw. 24(5), 493–500 (2011)
Calinon, S., Alizadeh, T., Caldwell, D.G.: On improving the extrapolation capability of task-parameterized movement models. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 610–616. IEEE (2013)
Kober, J., Peters, J.R.: Policy search for motor primitives in robotics. In: Advances in Neural Information Processing Systems, pp. 849–856 (2009)
Kober, J., Oztop, E., Peters, J.: Reinforcement learning to adjust robot movements to new situations. In: IJCAI Proceedings-International Joint Conference on Artificial Intelligence, vol. 22, p. 2650 (2011)
Stulp, F., Raiola, G., Hoarau, A., Ivaldi, S., Sigaud, O.: Learning compact parameterized skills with a single regression. In: 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids), pp. 417–422. IEEE (2013)
Nemec, B., Vuga, R., Ude, A.: Efficient sensorimotor learning from multiple demonstrations. Adv. Robot. 27(13), 1023–1031 (2013)
Mülling, K., Kober, J., Kroemer, O., Peters, J.: Learning to select and generalize striking movements in robot table tennis. Int. J. Robot. Res. 32(3), 263–279 (2013)
Ijspeert, A.J., Nakanishi, J., Schaal, S.: Movement imitation with nonlinear dynamical systems in humanoid robots. In: IEEE International Conference on Robotics and Automation, Proceedings, ICRA 2002, vol. 2, pp. 1398–1403. IEEE (2002)
Peters, J.R.: Machine learning of motor skills for robotics. Ph.D. thesis, University of Southern California (2007)
Kalakrishnan, M., Righetti, L., Pastor, P., Schaal, S.: Learning force control policies for compliant manipulation. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4639–4644. IEEE (2011)
Kalakrishnan, M., Chitta, S., Theodorou, E., Pastor, P., Schaal, S.: STOMP: stochastic trajectory optimization for motion planning. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 4569–4574. IEEE (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Lundell, J., Hazara, M., Kyrki, V. (2017). Generalizing Movement Primitives to New Situations. In: Gao, Y., Fallah, S., Jin, Y., Lekakou, C. (eds) Towards Autonomous Robotic Systems. TAROS 2017. Lecture Notes in Computer Science(), vol 10454. Springer, Cham. https://doi.org/10.1007/978-3-319-64107-2_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-64107-2_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-64106-5
Online ISBN: 978-3-319-64107-2
eBook Packages: Computer ScienceComputer Science (R0)