Incorporating decision-theoretic planning in a robot architecture

Abstract

The goal of robotics research is to design a robot to fulfill a variety of tasks in the real world. Inherent in the real world is a high degree of uncertainty about the robot’s behavior and about the world. We introduce a robot task architecture, DTRC, that generates plans with actions that incorporate costs and uncertain effects, and states that yield rewards.

The use of a decision-theoretic planner in a robot task architecture is demonstrated on the mobile robot domain of miniature golf. The miniature golf domain shows the application of decision-theoretic planning in an inherently uncertain domain, and demonstrates that by using decision-theoretic planning as the reasoning method in a robot task architecture, accommodation for uncertain information plays a direct role in the reasoning process.

Introduction

Robot task architectures represent an assemblage of algorithms that interpret user goals and sensor readings, in order to determine robot motor actions that will reach the user-specified goals. Many current approaches to robot task architectures use symbolic planning to perform high-level reasoning. This reasoning generates a sequence of actions for problems that transition the robot agent from an initial condition to a goal condition. In classical symbolic planning problems, the agent is given all relevant information about the world and executed actions are both deterministic and assumed to always succeed.

The use of a symbolic planner in a robot task architecture overlooks the inherent uncertainty in robotic domains. The uncertainty stems from the existence of multiple possible action effects, unreliable sensors, and incomplete domain information. Instead of having the planner address these issues, current task architectures choose to have other parts of the architecture handle problems arising from uncertainties or to create domain-specific and robot-specific workarounds that allow the robot to complete the given task.

To meet the goal for robots to complete tasks in the real world, the robot must deal with the uncertainty associated with actions, sensors and information. Additionally, actions in the real world have associated costs. To perform this reasoning within a robot architecture, we have developed the Decision-Theoretic Robot Controller (DTRC).

Our robot task control architecture contains three main elements: the DT-Graphplan planner, the robot skills, and the execution monitor. The DT-Graphplan planner is a decision-theoretic planner that generates a satisfying plan for domains with incomplete information, stochastic action effects, state reward conditions, and action costs. DT-Graphplan performs high-level reasoning over the domain, generating action sequences to transition the agent from the initial condition to the goal condition. The robot skills are a collection of low-level base building blocks that handle motor and sensor control and integration, essentially the actions the planner uses to assemble a plan. The execution monitor communicates between DT-Graphplan and the robot behaviors by maintaining current domain information for the planner and activating behaviors based on the plan provided. DTRC’s execution monitor also detects plan failure and triggers replanning from the current condition.

We illustrate DTRC on a robot miniature golf domain. For robot miniature golf, the task the robot must address is to get the golf ball into the cup with the fewest number of strokes. The course has a number of stationary and non-stationary obstacles to avoid. The robot has three methods of moving the ball, and each action has a different stroke penalty (cost) and outcome. Our results are generated from an implementation of this domain using a Pioneer 1 robot. This domain demonstrates the application of decision theory in a robot architecture, shows how changes in the domain result in changes in the preference of plans, and describes how the robot architecture sustains a probabilistic representation of the domain state and generates initial conditions for replanning.

This work introduces the DT-Graphplan planner and our robot control architecture, illustrating the application of the planner to imprecise and uncertain domains such as robotics. Section 2 briefly covers related robot control architecture work. Section 3 discusses the interactions of the elements of the task control architecture, and design of DT-Graphplan to handle decision theory instead of using a strictly Bayesian method. Section 4 contains the results of applying the architecture to the task of robot miniature golf, and the required elements to convert to the robot soccer domain and how altering the domain description changes the overall behavior of the robot. We finish with concluding remarks and possible extensions of this work.