Abstract
We propose a method that allows a motion planning algorithm to imitate the behavior of expert users in deformable environments. For instance, a surgeon inserting a probe knows intuitively which organs are more sensitive than others, but may not be able to mathematically represent a cost function that governs his or her motion. We hypothesize that the relative sensitivities of deformable objects are encoded in the expert’s demonstrated motion, and present a framework which is able to imitate an expert’s behavior by learning a sensitivity-based cost function under which the expert’s motion is optimal. Our framework consists of three stages: (1) Automatically generating demonstration tasks that prompt the user to provide informative demonstrations through an active learning process; (2) Recovering object sensitivity values using an Inverse Optimal Control technique; and (3) Reproducing the demonstrated behavior using an optimal motion planner. We have tested our framework with a set of 5DoF simulated and 3DoF physical test environments, and demonstrate that it recovers object parameters suitable for planning paths that imitate the behavior of expert demonstrations. Additionally, we show that our method is able to generalize to new tasks; e.g. when a new obstacle is introduced into the environment.
This work is supported in part by the Office of Naval Research under Grant N00014-13-1-0735 and by the National Science Foundation under Grant IIS-1317462.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
To determine which points are “inside” the environment, we compute a “local maxima map” using the Signed Distance Field (SDF) of the environment. For each point in the SDF, we follow the gradient away from obstacles and record the location the gradient becomes zero (i.e. the local distance maxima). Points “inside” the environment have corresponding local maxima inside the bounds of the SDF, while points “outside” have local maxima corresponding to the bounds of the SDF. Intuitively, “inside” points have finite-distance local maxima reachable via the gradient, while for “outside” points, the local maxima are undefined.
References
Abbeel, P., Ng, A.Y.: Apprenticeship learning via inverse reinforcement learning. In: ICML (2004)
Alterovitz, R., Goldberg, K.: Motion Planning in Medicine: Optimization and Simulation Algorithms for Image-Guided Procedures. Springer Tracts in Advanced Robotics. Springer, Berlin (2008)
Chen, L., Rong, Y.: Linear time recognition algorithms for topological invariants in 3D. In: International Conference on Pattern Recognition (2008)
Coumans, E.: Bullet 2.73 Physics SDK Manual (2010)
Faure, F., Gilles, B., Bousquet, G., Pai, D.K.: Sparse meshless models of complex deformable solids. In: ACM Transactions on Graphics pp. 73–73 (2011)
Gibson, S.F.F., Mirtich, B.: A survey of deformable modeling in computer graphics. Technical report, Mitsubishi Electric Research Laboratories (1997)
Irving, G., Teran, J., Fedkiw, R.: Invertible finite elements for robust simulation of large deformation. In: ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’04. New York, New York, USA (2004)
Jiménez, P.: Survey on model-based manipulation planning of deformable objects. Robot. Comput.-Integr. Manuf. 28(2), 154–163 (2012)
Kalakrishnan, M., Chitta, S., Theodorou, E., Pastor, P., Schaal, S.: STOMP: stochastic trajectory optimization for motion planning. In: ICRA (2011)
Kalakrishnan, M., Pastor, P., Righetti, L., Schaal, S.: Learning objective functions for manipulation. In: ICRA (2013)
Kalman, R.: When is a linear control system optimal? J. Basic Eng. (1964)
Karaman, S., Frazzoli, E.: Sampling-based algorithms for optimal motion planning. Int. J. Robot. Res. 30(7), 20 (2010)
Maris, B., Botturi, D., Fiorini, P.: Trajectory planning with task constraints in densely filled environments. In: IROS (2010)
Müller, M., Dorsey, J., McMillan, L., Jagnow, R., Cutler, B.: Stable real-time deformations. In: ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’02. New York, New York, USA (2002)
Ng, A.Y., Russell, S.J., et al.: Algorithms for inverse reinforcement learning. In: ICML (2000)
Phillips-Grafflin, C., Berenson, D.: A representation of deformable objects for motion planning with no physical simulation. In: ICRA (2014)
Ratliff, N., Silver, D., Bagnell, J.A.D.: Learning to search: functional gradient techniques for imitation learning. Auton. Robots (2009)
Şucan, I.A., Moll, M., Kavraki, L.E.: The open motion planning library. IEEE Robot. Autom. Mag. 19(4), 72–82 (2012). http://ompl.kavrakilab.org
Ziebart, B.D., et al.: Maximum Entropy Inverse Reinforcement Learning. In: AAAI, pp. 1433–1438 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this chapter
Cite this chapter
Phillips-Grafflin, C., Berenson, D. (2018). Reproducing Expert-Like Motion in Deformable Environments Using Active Learning and IOC. In: Bicchi, A., Burgard, W. (eds) Robotics Research. Springer Proceedings in Advanced Robotics, vol 2. Springer, Cham. https://doi.org/10.1007/978-3-319-51532-8_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-51532-8_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-51531-1
Online ISBN: 978-3-319-51532-8
eBook Packages: EngineeringEngineering (R0)