Skip to main content
SpringerLink
Log in

One-shot learning of human–robot handovers with triadic interaction meshes

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

We propose an imitation learning methodology that allows robots to seamlessly retrieve and pass objects to and from human users. Instead of hand-coding interaction parameters, we extract relevant information such as joint correlations and spatial relationships from a single task demonstration of two humans. At the center of our approach is an interaction model that enables a robot to generalize an observed demonstration spatially and temporally to new situations. To this end, we propose a data-driven method for generating interaction meshes that link both interaction partners to the manipulated object. The feasibility of the approach is evaluated in a within user study which shows that human–human task demonstration can lead to more natural and intuitive interactions with the robot.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. For each agent we capture feet, hands, elbows, head and pelvis.

  2. Distances below \(\varOmega =0.75\)cm are considered to be relevant for the interaction.

  3. The transformations of \({\mathbf {p}}_{1:T}\) into different coordinate systems can be achieved with a single matrix operation and we will henceforth omit a precise marking for reasons of readability. Instead we reference the corresponding space at each occurrence.

  4. For (13), \(Z \times Z \) constraint matrices C, F, and W are transformed into \(3Z \times 3Z\) shape to account for each Cartesian component of the joint positions separately. The \(3Z \times 3Z\) diagonal matrices are created as in (8) but with each element being repeated three times.

  5. 3 recordings were dropped due to inconsistencies in motion capture readings.

References

  • Admoni, H., Dragan, A., Srinivasa, S, & Scassellati, B. (2014). Deliberate delays during robot-to-human handovers improve compliance with gaze communication. In International conference on human-robot interaction (HRI).

  • Bankó, Z., & Abonyi, J. (2012). Correlation based dynamic time warping of multivariate time series. Expert Systems with Applications, 39(17), 12814–12823.

    Article  Google Scholar 

  • Ben Amor, H. (2010). Imitation learning of motor skills for synthetic humanoids. Ph.D. dissertation, Technische Universität Bergakademie Freiberg.

  • Ben Amor, H., Neumann, G., Kamthe, S., Kroemer, O., & Peters, J. (2014). Interaction primitives for human-robot cooperation tasks. In 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE, May 2014 (pp. 2831–2837).

  • Berndt, D. J., & Clifford, J. (1994). Using dynamic time warping to find patterns in time series. In Knowledge Discovery in Databases (pp. 359–370). Washington: Seattle.

  • Dehais, F., Sisbot, E. A., Alami, R., & Causse, M. (2011). Physiological and subjective evaluation of a humanrobot object hand-over task. Applied Ergonomics, 42(6), 785–791.

    Article  Google Scholar 

  • Dragan, A., Lee, K., & Srinivasa, S. (2013). Legibility and predictability of robot motion. Human-Robot Interaction.

  • Duvallet, F., Karnam, M., & Billard, A. (2016) . A human-inspired controller for fluid human-robot handovers. In Humanoids 2016—16th IEEE-RAS international conference on humanoid robots (pp. 324–331).

  • Ewerton, M., Neumann, G., Lioutikov, R., Amor, H. B. , Peters, J., & Maeda, G. (2015). Learning multiple collaborative tasks with a mixture of interaction primitives. In IEEE International conference on robotics and automation (pp. 1535–1542).

  • Ho, E. S. L., Chan, J. C. P., Komura, T., & Leung, H. (2013). Interactive partner control in close interactions for real-time applications. ACM Transactions on Multimedia Computing, Communications, and Applications, 9(3), 1–19.

    Article  Google Scholar 

  • Ho, E. S. L., Komura, T., & Tai, C.-L. (2010). Spatial relationship preserving character motion adaptation. ACM Transactions on Graphics, 29(4), 1.

    Article  Google Scholar 

  • Huang, C.-M., Cakmak, M., & Mutlu, B. (2015). Adaptive coordination strategies for human-robot handovers. In Robotics: Science and Systems XI. Robotics: Science and Systems Foundation, July.

  • Ivan, V., Zarubin, D., Toussaint, M., Komura, T., & Vijayakumar, S. (2013). Topology-based representations for motion planning and generalization in dynamic environments with interactions. The International Journal of Robotics Research, 32(9–10), 1151–1163.

    Article  Google Scholar 

  • Kupcsik, A., Hsu, D., & Lee, W. S. (2016). Learning dynamic robot-to-human object handover from human feedback. CoRR, (Vol. abs/1603.06390, 2016) (Online). Available: http://arxiv.org/abs/1603.06390

  • Lee, D., Ott, C., & Nakamura, Y. (2010). Mimetic Communication Model with Compliant Physical Contact in Human-Humanoid Interaction. The International Journal of Robotics Research, 29(13), 1684–1704.

    Article  Google Scholar 

  • Mainprice, J., Gharbi, M., Siméon, T., & Alami, R. (2012). Sharing effort in planning human-robot handover tasks. In IEEE Ro-man: The 21st IEEE International Symposium on Robot and Human Interactive Communication. IEEE, 2012 (pp. 764–770).

  • Quispe, A. H., Ben Amor, H., & Stilman, M. (2014). Handover planning for every occasion. In 2014 IEEE-RAS International Conference on Humanoid Robots. IEEE, November 2014 (pp. 431–436). (Online) Available: http://ieeexplore.ieee.org/document/7041396/

  • Schaal, S. (1999). Is Imitation Learning the Route to Humanoid Robots? Trends in Cognitive Sciences, 3(6), 233–242.

    Article  Google Scholar 

  • Sorkine, O., Cohen-Or, D., Lipman, Y., Alexa, M., Rössl, C., & Seidel, H.-P. (2004) . Laplacian surface editing. In Proceedings of the EUROGRAPHICS/ACM SIGGRAPH Symposium on Geometry Processing. ACM Press (pp. 179–188).

  • Strabala, K., Lee, M. K., Dragan, A., Forlizzi, J., Srinivasa, S. S., Cakmak, M., et al. (2013). Toward Seamless Human-robot handovers. Journal of Human-Robot Interaction, 2(1), 112–132.

  • Vogt, D., Lorenz, B., Grehl, S., & Jung, B. (2015). Behavior generation for interactive virtual humans using context-dependent interaction meshes and automated constraint extraction. Computer Animation and Virtual Worlds, 26(3–4), 227–235.

    Article  Google Scholar 

  • Yang, Y., Ivan, V., & Vijayakumar, S. (2015). Real-time motion adaptation using relative distance space representation. In International Conference on Advanced Robotics (ICAR), No. ICAR. IEEE, July 2015 (pp. 21–27).

  • Zheng, M., Moon, A., Croft, E. A., & Meng, M. Q. (2015). Impacts of robot head gaze on robot-to-human handovers. International Journal of Social Robotics, 7(5), 783–798.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Mathematics and Informatics, Technical University Bergakademie Freiberg, 09599, Freiberg, Germany

    David Vogt & Bernhard Jung

  2. School of Computing, Informatics, Decision Systems Engineering, Arizona State University, 699 S Mill Ave, Tempe, AZ, 85281, USA

    Simon Stepputtis & Heni Ben Amor

Authors
  1. David Vogt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Stepputtis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bernhard Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Heni Ben Amor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Vogt.

Additional information

This is one of the several papers published in Autonomous Robots comprising the Special Issue on Learning for Human–Robot Collaboration.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vogt, D., Stepputtis, S., Jung, B. et al. One-shot learning of human–robot handovers with triadic interaction meshes. Auton Robot 42, 1053–1065 (2018). https://doi.org/10.1007/s10514-018-9699-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-018-9699-4

Keywords

Search

Navigation