Robot learning through task identification

Abstract

The operation of an autonomous mobile robot in a semi-structured environment is a complex, usually non-linear and partly unpredictable process. Lacking a theory of robot–environment interaction that allows the design of robot control code based on theoretical analysis, roboticists still have to resort to trial-and-error methods in mobile robotics.

The RobotMODIC project aims to develop a theoretical understanding of a robot’s interaction with its environment, and uses system identification techniques to identify the system robot–task–environment. In this paper, we present two practical examples of the RobotMODIC process: mobile robot self-localisation and mobile robot training to achieve door traversal.

In both examples, a transparent mathematical function is obtained that maps inputs–sensory perception in both cases–to output — location and steering velocity respectively. Analysis of the obtained models reveals further information about the way in which a task is achieved, the relevance of individual sensors, possible ways of obtaining more parsimonious models, etc.

Introduction

The behaviour of a mobile robot is the result of properties of the robot itself (physical aspects–the “embodiment”), the environment (“situatedness”), and the control program (the “task”) the robot is executing (see Fig. 1). This triangle of robot, task and environment constitutes a complex, interacting and typically nonlinear system that is difficult to analyse, and whose behaviour is only partially predictable. Because of this lack of theoretical understanding, it is hard to develop robot control code without resorting to costly and imprecise trial-and-error methods.

The RobotMODIC project [1] investigates this relationship between robot, task and environment, and aims to develop an analytical description of the interdependency between the factors that govern robot behaviour. The approach we have taken so far is to model aspects of robot behaviour, using transparent mathematical models which, we argue, retain the essential properties of the robot’s behaviour, while being analysable through established mathematical procedures.

Essentially, we view the interaction between a mobile robot and its environment as a kind of “computation”, in which the robot “computes” behaviour (the output) from the three inputs: robot morphology, environmental characteristics and executed task (see Fig. 2).

Similar to a cylindrical lens, which can be used to perform an analog computation, highlighting vertical edges and suppressing horizontal ones, or a camera lens computing a Fourier transform by analog means, a robot’s behaviour–commonly the mobile robot’s trajectory–can be seen as emergent from the three components shown in Fig. 1: the robot “computes” its behaviour from its own makeup, the world’s makeup, and taking into account the program it is currently running (the task).

The RobotMODIC procedure, therefore, consists of four main steps:

• Observe the robot’s behaviour in the target environment, by logging sensory perception, motor response and location in the environment,

• model the relationship under investigation, using a transparent mathematical modelling process that expresses the relationship as a closed, analysable function,

• establish the validity of the obtained model by comparing the originally observed behaviour with that of the model,

• analyse the model using standard mathematical techniques, thus gaining understanding of the interaction between robot and environment.

To obtain models of input–output relationships we use the ARMAX (linear autoregressive, moving average model with exogenous inputs, [2]) or NARMAX (nonlinear ARMAX) system identification methods; we therefore refer to this process as “robot identification”.

Both ARMAX and NARMAX model the input–output relationship as a mathematical function, in our case linear or nonlinear polynomials. ARMAX is available through standard programming packages such as Matlab or Scilab.

The NARMAX modelling approach is a parameter estimation methodology for identifying both the important model terms and the parameters of unknown non-linear dynamic systems. For multiple input, single output noiseless systems this model takes the form:

$$y\left(n\right)=f(u_1\left(n\right),u_1(n-1),u_1(n-2),\dots ,u_1(n-N_u),u_1{\left(n\right)}^2,u_1{(n-1)}^2,u_1{(n-2)}^2,\dots ,u_1{(n-N_u)}^2,\dots ,u_1{\left(n\right)}^l,u_1{(n-1)}^l,u_1{(n-2)}^l,\dots ,u_1{(n-N_u)}^l,u_2\left(n\right),u_2(n-1),u_2(n-2),\dots ,u_2(n-N_u),u_2{\left(n\right)}^2,u_2{(n-1)}^2,u_2{(n-2)}^2,\dots ,u_2{(n-N_u)}^2,\dots ,u_2{\left(n\right)}^l,u_2{(n-1)}^l,u_2{(n-2)}^l,\dots ,u_2{(n-N_u)}^l,\dots ,\dots ,u_d\left(n\right),u_d(n-1),u_d(n-2),\dots ,u_d(n-N_u),u_d{\left(n\right)}^2,u_d{(n-1)}^2,u_d{(n-2)}^2,\dots ,u_d{(n-N_u)}^2,\dots ,u_d{\left(n\right)}^l,u_d{(n-1)}^l,u_d{(n-2)}^l,\dots ,u_d{(n-N_u)}^l,y(n-1),y(n-2),\dots ,y(n-N_y),y{(n-1)}^2,y{(n-2)}^2,\dots ,y{(n-N_y)}^2,\dots ,y{(n-1)}^l,y{(n-2)}^l,\dots ,y{(n-N_y)}^l)$$

where $y\left(n\right)$ and $\overrightarrow{u}\left(n\right)$ are the sampled output and input signals at time $n$ respectively, $N_y$ and $N_u$ are the regression orders of the output and input respectively and $d$ is the input dimension. $f\left(\right)$ is a non-linear function and it is typically taken to be a polynomial or wavelet multi-resolution expansion of the arguments. The degree $l$ of the polynomial is the highest sum of powers in any of its terms.

The NARMAX methodology breaks the modelling problem into the following steps:

• Structure detection,

• parameter estimation,

• model validation,

• prediction, and

• analysis.

A detailed procedure of how these steps are done is presented in [3], [4], [5]; nevertheless a brief explanation of them is given below.

Any data set that we intend to model is first split in two sets (usually of equal size). The first, referred to as the estimation data set, is used to calculate the model parameters. The remaining data set, referred to as the test set, is used to test and evaluate the model.

The structure of the NARMAX polynomial is determined by the inputs $\overrightarrow{u}$, the output $y$, the input and output orders $N_u$ and $N_y$ respectively and the degree $l$ of the polynomial. The number of terms of the NARMAX model polynomial can be very large depending on these variables, but not all of them are significant contributors to the computation of the output, in fact often most terms can be safely removed from the model equation without introducing any significant errors.

The calculation of the NARMAX model parameters is an iterative process. Each iteration involves three steps: (i) estimation of model parameters, (ii) model validation and (iii) removal of non-contributing terms.

In the first step the NARMAX model is used to compute an equivalent auxiliary model whose terms are orthogonal, thus allowing their associated parameters to be calculated sequentially and independently from each other. Once the parameters of the auxiliary model are obtained, the NARMAX model parameters are computed from the auxiliary model.

The NARMAX model is then tested using the validation data set. If the error between the model predicted output and the actual output is below a user-defined threshold, non-contributing model terms are removed in order to reduce the size of the polynomial.

To determine the contribution of a model term to the output, the so-called Error Reduction Ratio (ERR) [4] is computed for each term. The ERR of a term is the percentage reduction in the total mean-squared error (i.e. the difference between model predicted and true system output) as a result of including (in the model equation) the term under consideration. The bigger the ERR is, the more significant the term. Model terms with ERR under a certain threshold are removed from the model polynomial.

In the following iteration, if the error is higher as a result of the last removal of model terms then these are re-inserted back into the model equation and the model is considered as final.

The RobotMODIC process offers some very practical benefits to the robotics engineer. It is possible, for instance, to represent robot control code in a very condensed and “universal” format that allows the easy transfer of robot control code between different robot platforms [6] (see, for instance, Section 3 of this paper). It is also possible to represent one sensor modality in terms of another, for example sonar sensor perceptions can be expressed as functions of laser perceptions (this “sensor identification” is obviously only possible if both sensor modalities contain similar information — to model laser perception as a function of bumper signals, for example, is of course impossible! [7]).

In this paper, we present two practical examples of how this process can be applied to mobile robotics:

• Robot navigation: self-localisation of a mobile robot.

• Robot programming: obtaining robot control code through operator-supervised training.