Learning object deformation models for robot motion planning

Highlights

• We present a planning system for robots in environments with deformable objects.

• A manipulation robot determines the deformation parameters of real objects.

• We consider the costs of object deformations by finite element simulations.

• The deformation costs are modeled using Gaussian processes for efficient planning.

• Application to wheeled and manipulation robots operating in real environments.

Abstract

In this paper, we address the problem of robot navigation in environments with deformable objects. The aim is to include the costs of object deformations when planning the robot’s motions and trade them off against the travel costs. We present our recently developed robotic system that is able to acquire deformation models of real objects. The robot determines the elasticity parameters by physical interaction with the object and by establishing a relation between the applied forces and the resulting surface deformations. The learned deformation models can then be used to perform physically realistic finite element simulations. This allows the planner to evaluate robot trajectories and to predict the costs of object deformations. Since finite element simulations are time-consuming, we furthermore present an approach to approximate object-specific deformation cost functions by means of Gaussian process regression. We present two real-world applications of our motion planner for a wheeled robot and a manipulation robot. As we demonstrate in real-world experiments, our system is able to estimate appropriate deformation parameters of real objects that can be used to predict future deformations. We show that our deformation cost approximation improves the efficiency of the planner by several orders of magnitude.

Introduction

Perceiving the surroundings and modeling the environment is an important competence of intelligent mobile robots since such models are required for efficiently solving other high-level tasks. For instance, generating a collision-free path through the environment in an efficient way requires path planning, which in turn builds on top of a model of the environment. There exists a variety of approaches for robots to autonomously generate an appropriate model of the environment by addressing the simultaneous localization and mapping problem  [1], [2], [3], [4], by autonomous exploration  [5], [6], [7], or by addressing both problems jointly  [8], [9].

In order to plan motions in learned environment models, the majority of path planning approaches assumes that the environment contains only rigid obstacles  [10], [11], [12], although there are a few notable exceptions such as the works of  [13], [14], [15], [16], [17]. In reality, not all obstacles are rigid. In domestic environments–a key target domain for service robots–a robot must deal with many deformable objects such as plants, curtains, or cloth. Considering that an object such as a curtain is deformable can enable a robot to accomplish navigation tasks that otherwise cannot be carried out.

To consider deformable objects in the path planning process, such objects need to be handled in a simulation system underlying the planner. The realistic simulation of object deformations is still an active area of research with a variety of relevant applications in computer graphics, virtual reality, games, movies, but also in robotics  [18], [19], and medical simulations  [20], [21], [22]. In most applications, the underlying parameters for an appropriate deformation simulation are adjusted manually until the results appear visually plausible. This might be applicable for computer games or movies, but does not necessarily lead to a physically realistic computation of the involved forces. These forces, however, need to be known accurately for navigation in the presence of deformable objects. For example, whenever a robot interacts with real-world objects, only limited forces should be applied to them. This is of utmost importance in medical applications or in domestic settings, for instance, whenever robots have to manipulate plants or clothes. Particularly in these domains, robots need exact knowledge about the parameters of the deformation process.

In this paper, we present a complete robotic system that is able to perceive the environment and model the deformable objects in the scene. The system estimates the deformation properties of objects, and finally is able to plan a trajectory through the environment, taking potential object deformations into account.

Estimating the elasticity parameters of objects not only involves observing and reconstructing the three-dimensional surface of an object. Physical interaction with the object under consideration is required to learn about its behavior when exposed to external forces. Therefore, we equipped our robot with a force sensor at the end of the manipulator and with a depth camera. Based on the observed surface deformations and corresponding forces, our approach seeks to determine the elasticity parameters of the object. This is done by simulating the object deformation under the applied forces using a linear finite element model. An error minimization approach is applied to iteratively adapt the deformation parameters such that the difference between the real object under deformation and the simulation is minimized. As we will demonstrate in the experimental section of this paper, our approach is able to find elasticity parameters that enable our robot to accurately predict the deformations of real-world objects.

Furthermore, we address the problem of planning motions for robots navigating in environments with deformable obstacles and to adequately consider the costs of object deformations. In this context, we present an efficient approximation of the deformation cost function of objects. Throughout this paper, we assume that the robot can deform the objects but does not move them in the environment. This allows us to generate a set of trajectory samples in a pre-processing step and to predict the costs of new trajectories by applying efficient Gaussian process regression. Using this regression approach, the robot is able to efficiently plan trajectories in the presence of deformable objects without the need for time-consuming simulations during runtime. In different experiments, we demonstrate that our approach yields accurate estimates and, at the same time, allows for efficient planning of trajectories along which the robot interacts with deformable objects.

This paper is organized as follows: after discussing related work in the next section, we will give an overview of our planning approach that considers deformable objects and describe the basic principles of the deformation model and the physical simulation underlying our planner in Section  3. In Section  4, we describe how to learn models of deformable objects with our manipulation robot. Next, we present our approach to approximate the deformation cost functions of objects using Gaussian process regression in Section  5. Subsequently, we present two applications of our path planning system applied to a manipulation robot and a wheeled robot. Finally, in Section  7, we evaluate our system in different experiments.