Skip to main content

Advertisement

SpringerLink
Log in

Learning of Planning Models for Dexterous Manipulation Based on Human Demonstrations

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

In the human environment service robots have to be able to manipulate autonomously a large variety of objects in a workspace restricted by collisions with obstacles, self-collisions and task constraints. Planning enables the robot system to generalize predefined or learned manipulation knowledge to new environments. For dexterous manipulation tasks the manual definition of planning models is time-consuming and error-prone. In this work, planning models for dexterous tasks are learned based on multiple human demonstrations using a general feature space including automatically generated contact constraints, which are automatically relaxed to consider the correspondence problem. In order to execute the learned planning model with different objects, the contact location is transformed to given object geometry using morphing. The initial, overspecialized planning model is generalized using a previously described, parallelized optimization algorithm with the goal to find a maximal subset of task constraints, which admits a solution to a set of test problems. Experiments on two different, dexterous tasks show the applicability of the learning approach to dexterous manipulation tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 13
Fig. 14
Fig. 15
Fig. 17
Fig. 19
Fig. 20
Fig. 22
Fig. 23

Similar content being viewed by others

Notes

  1. By using the quaternion representation, a rotation matrix can be represented as a single rotation around a fixed axis with a fixed angle. The scaled axis representation is the product of this axis and angle.

References

  1. Azad P, Gockel T, Dillmann R (2007) Computer vision—das praxisbuch. Elektor, Aachen

    Google Scholar 

  2. Berenson D, Srinivasa S, Ferguson D, Kuffner J (2009) Manipulation planning on constraint manifolds. In: IEEE int conf on robotics and automation

    Google Scholar 

  3. Breazeal C, Berlin M, Brooks AG, Gray J, Thomaz AL (2006) Using perspective taking to learn from ambiguous demonstrations. Robot Auton Syst 54(5):385–393

    Article  Google Scholar 

  4. Calinon S, Billard A (2006) Teaching a humanoid robot to recognize and reproduce social cues. In: Proceedings of the IEEE international symposium on robot and human interactive communication (RO-MAN), pp 346–351

    Chapter  Google Scholar 

  5. Calinon S, Guenter F, Billard A (2005) Goal-directed imitation in a humanoid robot. In: Proceedings of the IEEE international conference on robotics and automation

    Google Scholar 

  6. Calinon S, Guenter F, Billard A (2007) On learning, representing, and generalizing a task in a humanoid robot. IEEE Trans Syst Man Cybern, Part B, Cybern 37(2):286–298

    Article  Google Scholar 

  7. Eiben A, Smith J (2003) Introduction to evolutionary computing. Springer, Berlin

    MATH  Google Scholar 

  8. Erlhagen W, Mukovskiy A, Bicho E, Panin G, Kiss C, Knoll A, van Schie HT, Bekkering H (2006) Goal-directed imitation for robots: a bio-inspired approach to action understanding and skill learning. Robot Auton Syst 54(5):353–360

    Article  Google Scholar 

  9. Friedrich H, Dillmann R (1995) Robot programming based on a single demonstration and user intentions. In: 3rd European workshop on learning robots at ECML’95

    Google Scholar 

  10. Friedrich H, Dillmann R, Rogalla O (1999) Interactive robot programming based on human demonstration and advice. In: Selected papers from the international workshop on sensor based intelligent robots. Springer, London, pp 96–119

    Google Scholar 

  11. Hu H, Gao X, Li J, Wang J, Liu H (2004) Calibrating human hand for teleoperating the hit/dlr hand. In: IEEE international conference on robotics and automation, pp 4571–4576

    Google Scholar 

  12. Ijspeert AJ, Nakanishi J, Schaal S (2002) Movement imitation with nonlinear dynamical systems in humanoid robots. In: IEEE int conf on robotics and automation. IEEE Press, New York, pp 1398–1403

    Google Scholar 

  13. Jäkel R, Schmidt-Rohr SR, Loesch M, Dillmann R (2010) Representation and constrained planning of manipulation strategies in the context of programming by demonstration. In: IEEE international conference on robotics and automation (ICRA’10)

    Google Scholar 

  14. Jäkel R, Schmidt-Rohr SR, Loesch M, Kasper A, Dillmann R (2010) Learning of generalized manipulation strategies in the context of programming by demonstration. In: 10th IEEE-RAS international conference on humanoid robots (Humanoids)

    Google Scholar 

  15. Jäkel R, Meißner P, Schmidt-Rohr SR, Dillmann R (2011) Distributed generalization of learned planning models in robot programming by demonstration. In: IEEE/RSJ int conf on intelligent robots and systems

    Google Scholar 

  16. Kasper A, Xue Z, Dillmann R (2012) A object model database for object recognition, localisation and manipulation in service robotics. Int J Robot Res. doi:10.1177/0278364912445831

  17. Keijzer M, Merelo JJ, Romero G, Schoenauer M (2002) Evolving objects: a general purpose evolutionary computation library. Artif Evol 2310:829–888

    Google Scholar 

  18. Kim V, Lipman Y, Funkhouser T (2011) Blended intrinsic maps. ACM Trans Graph 30:4. doi:10.1145/2010324.1964974

    Google Scholar 

  19. Kuniyoshi Y, Inaba M, Inoue H (1994) Learning by watching: extracting reusable task knowledge from visual observation of human performance. IEEE Trans Robot Autom 10:799–822

    Article  Google Scholar 

  20. Muhlig M, Gienger M, Steil J, Goerick C (2009) Automatic selection of task spaces for imitation learning. In: IEEE/RSJ international conference on intelligent robots and systems, IROS 2009, pp 4996–5002

    Chapter  Google Scholar 

  21. Pardowitz M, Glaser B, Dillmann R (2007) Learning repetitive robot programs from demonstrations using version space algebra. In: Proceedings of the 13th IASTED international conference on robotics and applications. ACTA Press, Anaheim, pp 394–399

    Google Scholar 

  22. Pastor P, Hoffmann H, Asfour T, Schaal S (2009) Learning and generalization of motor skills by learning from demonstration. In: IEEE international conference on robotics and automation

    Google Scholar 

  23. Pastor P, Kalakrishnan M, Chitta S, Theodorou E, Schaal S (2011) Skill learning and task outcome prediction for manipulation. In: IEEE international conference on robotics and automation

    Google Scholar 

  24. Rybski P, Yoon K, Stolarz J, Veloso M (2007) Interactive robot task training through dialog and demonstration. In: 2nd ACM/IEEE international conference on human-robot interaction

    Google Scholar 

  25. Siciliano B, Villani L, Oriolo G, Sciavicco L (2008) Robotics. Springer, New York

    MATH  Google Scholar 

  26. Steffen J, Elbrechter C, Haschke R, Ritter H (2010) Bio-inspired motion strategies for a bimanual manipulation task. In: 10th IEEE-RAS international conference on humanoid robots (Humanoids), pp 625–630

    Chapter  Google Scholar 

  27. Tang M, Kim Y, Manocha D (2009) C2a: controlled conservative advancement for continuous collision detection of polygonal models. In: IEEE international conference on robotics and automation (ICRA’09). IEEE Press, New York, pp 849–854

    Chapter  Google Scholar 

  28. van Lent M, Laird JE (2001) Learning procedural knowledge through observation. In: Proceedings of the 1st international conference on knowledge capture, K-CAP ’01. ACM Press, New York, pp 179–186

    Chapter  Google Scholar 

  29. Veeraraghavan H (2008) Teaching sequential tasks with repetition through demonstration (short paper). In: Proceedings of the international conference on autonomous agents and multiagent systems, AAMAS’08

    Google Scholar 

Download references

Acknowledgements

This work has been partially conducted within the German SFB 588 “Humanoid Robots” granted by DFG and within the ECs Integrated Project DEXMART under grant agreement no. 126239 (FP7/2007-2013).

Author information

Authors and Affiliations

  1. Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany

    Rainer Jäkel

  2. Karlsruhe Institute of Technology, Karlsruhe, Germany

    Sven R. Schmidt-Rohr, Alexander Kasper & Rüdiger Dillmann

  3. FZI Forschungszentrum Informatik, Haid-und-Neu-Str. 10-14, 76131, Karlsruhe, Germany

    Steffen W. Rühl & Zhixing Xue

Authors
  1. Rainer Jäkel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sven R. Schmidt-Rohr
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Steffen W. Rühl
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alexander Kasper
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhixing Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rüdiger Dillmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rainer Jäkel.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jäkel, R., Schmidt-Rohr, S.R., Rühl, S.W. et al. Learning of Planning Models for Dexterous Manipulation Based on Human Demonstrations. Int J of Soc Robotics 4, 437–448 (2012). https://doi.org/10.1007/s12369-012-0162-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-012-0162-y

Keywords

Search

Navigation