Considerations in the application of evolution to the generation of robot controllers

https://doi.org/10.1016/S0020-0255(01)00081-0Get rights and content

Abstract

This paper is concerned with different aspects of the use of evolution for the successful generation of real robot Artificial Neural Network (ANN) controllers. Several parameters of an evolutionary/genetic algorithm (GA) and the way they influence the evolution of ANN behavioral controllers for real robots have been contemplated. These parameters include the way the initial populations are distributed, how the individuals are evaluated, the implementation of race schemes, etc. A batch of experiments on the evolution of three types of behaviors with different population sizes have been carried out in order to ascertain their effect on the evolution of the controllers and their validity in real implementations. The results provide a guide to the design of evolutionary algorithms for generating ANN based robot controllers, especially when, due to computational constraints, the populations to be used are small with respect to the complexity of the problem to be solved. The problem of transferring the controllers evolved in simulated environments to the real systems operating in real environments are also considered and we present results of this transference to reality with a robot which has few and extremely noisy sensors.

Introduction

In the late 1980s and early 1990s, some authors, such as Harvey et al. [7], Cliff et al. [4] and Beer and Gallagher [2], have proposed artificial evolution as means to automate the design procedure of behavior based robot controllers. Many authors have taken up this issue and have developed different evolutionary mechanisms and strategies in order to obtain controllers for the autonomous operation of robots in structured and unstructured environments. Most of the algorithms employed were specific versions for the problems in hand, and, in the literature, there are very few data which can be taken as general or even helpful for contemplating the design of Artificial Neural Network (ANN) based robot controller evolvers.

The most usual structures employed for the implementation of behavior controllers are ANNs [2], [7], [15], [18]. ANNs form a powerful distributed processing model with very interesting characteristics for their use in autonomous robotics, such as their fault tolerance, noise tolerance (crucial property because of the presence of noise in real environments) and the possibility of using the traditional connectionist learning algorithms for obtaining the ANN in ontogenetic time or the possibility of combination of evolution with learning. Another advantage is derived from the fact that the weights and nodes are low level syntactic primitives, well below the semantic level [3], and, as Nolfi et al. [15] argue, the primitives considered by the evolutionary process must be of the lowest level possible so that undesirable selections produced by a human designer are avoided.

Evolution of ANNs as robot controllers imply a series of problems: on one hand, the evolution of ANN based structures is very prone to epistasis problems; on the other hand, the evaluation of the controllers is necessarily very indirect, as one does not know how good a controller is until the robot has interacted with the environment for a relatively long period of time. As a consequence, the computational load falls mainly on the simulation of the live of each individual in its environment, generally forcing the designer to work with small populations of individuals so as to make the evolutionary process as efficient as possible. This obviously leads to premature convergence problems, where, because of the small number of individuals in the populations, it is very easy for one individual to dominate and lead to a population consisting of replicas of this individual.

Some authors, such as Salomon [18], have employed the evolutionary strategy alternative because it aids in problems where the level of epistasis, and consequently, of deceptivity is high. That is because the fitness of a gene within the chromosome is strongly dependent on the value of other genes and, in this case, where the chromosomes represent the weights or other distributed parameters of ANNs, there is usually a large correlation between the genes in terms of fitness. The general trend, however, has been to make use of the classical genetic algorithm (GA) [4], [6], [7], [16] or small variations of it, and most authors have concentrated mostly on different aspects of the adaptation of the evaluation function, without really going into other parameters affecting the evolutionary algorithm.

In this paper we are going to address some of the solutions adopted by our group for the successful generation of controllers for real robots operating in real environments and which have been integrated in the Simulation and Evolution Environment (SEVEN), our general purpose automatic robot controller generator. To reach these conclusions, several experiments were carried out in the generation of controllers for three tasks considering different values for the parameters that influence the evolutionary process.

Section snippets

Experimental setup

In the examples we present throughout this article, the robot employed is a “Rug Warrior”. It is a small (18.5 cm diameter), simple and cheap circular robot. It has two DC motors, two infrared emitters and one binary receiver, two photosensors, one pyrosensor, three bumper sensors and two optical encoders. The sensors and actuators are very low quality, very noisy and imprecise, which is a plus when trying to obtain robust behaviors in the hardest possible conditions. The response of these

Distribution of the initial population in the solution space

Generally, the starting population of an evolutionary process is initiated randomly. Thus, if the number of individuals is not very large, the individuals may cluster around certain locations in the search space, with no individuals searching other areas. This is a typical problem of evolutionary robotics, where the number of individuals is usually small due to computing constraints.

To address this problem, we have considered different distribution strategies. The initial individuals were

Mutation and crossover

Two of the most studied operators with a bearance on evolutionary algorithms are the mutation operator and the crossover operator. Here, we have considered their relative influence on the results provided by the behavior generator. In the tests performed, and in order to maintain consistency, the sum of the crossover probability (Pc) and the mutation probability (Pm) is 1.

In evolutionary strategies there are several ways of approaching mutation. It is possible to fix the maximum mutation

Fitness evaluation

Whatever the evolution mechanism employed, the controllers corresponding to the different individuals must be evaluated and their fitness obtained. This fitness corresponds to how well the robot performs in an environment during its lifetime. Two different perspectives are possible from the point of view of evolutionary robotics: a local perspective or a global perspective. The first one is to establish for each step of the robot life a goodness of its actions in relation to its goal. The final

Noise in simulation

For a simulation to be appropriate for the transference of behaviors developed using it to the real world, it must meet some criteria, such as those established by Jakobi et al. [10], which usually imply handling different levels and types of noise. In the evolutionary robotics literature several authors have proposed different types of noise that should be applied to the simulated sensors, actuators or environment. Traditionally, the noise applied to the simulated sensors and actuators

Conclusions

In this work we have studied the impact of different parameters of an evolutionary algorithm on the performance of ANN based robot controllers obtained using it. These parameters included the crossover probability, going from an evolutionary strategy to a completely GA, the distribution of the initial populations, both in terms of the initial values for the genes making up the chromosomes and in terms of the distribution of the population into different races. The results indicate that for

Acknowledgements

This work was funded by the Xunta de Galicia under project PGIDT99PXI10503A and MCYT under TIC2000-0739-C04-04.

References (19)

  • D. Floreano et al.

    Evolutionary neurocontrollers for autonomous mobile robots

    Neural Networks

    (1998)
  • J.A. Becerra, J. Santos, R.J. Duro, Progressive construction of compound behavior controllers for autonomous robots...
  • R.D. Beer et al.

    Evolving dynamical neural networks for adaptive behavior

    Adaptive Behavior

    (1992)
  • D.J. Chalmers, Subsymbolic computation and the chinese room, in: J. Dinsmore (Ed.), Symbolic and Connectionist...
  • D. Cliff et al.

    Evolving visually guided robots

  • J.C. Gallagher, R.D. Beer, K.S. Espenschied, R.D. Quinn, Application of evolved locomotion controllers to a Hexapod...
  • I. Harvey et al.

    Issues in evolutionary robotics

  • J. Holland

    Adaptation in Natural and Artificial Systems

    (1975)
  • P. Husbands, I. Harvey, D. Cliff, A. Thompson, N. Jakobi, The artificial evolution of control systems, in: I.C. Parmee...
There are more references available in the full text version of this article.

Cited by (13)

  • Evolvability signatures of generative encodings: Beyond standard performance benchmarks

    2015, Information Sciences
    Citation Excerpt :

    Since the eighties, the ER field has made amazing progress to both design sophisticated artifacts and to endow machines with impressive adaptive abilities. For instance, it allows for the automated construction of modular, three-dimensional, physically locomoting robots, [48], to synthesize neural networks to control robot behaviors (e.g., [69,72,84,99]), and discover a multitude of walking gaits for multilegged robots following unforeseen mechanical damages [10,22,66]. However, even the most advanced evolved artifacts are still far behind the state of the art in mainstream robotics [11,59]: conventionally engineered robots are capable of operating successfully in a wide variety of indoor and outdoor environments (e.g., locomotion with the BigDog quadruped robot, [91]), whereas the best evolved robots are still only capable of simplistic behaviors (e.g., walking in a straight line on a flat terrain, or avoiding obstacles in an enclosed indoor arena).

  • On the potential contributions of hybrid intelligent approaches to Multicomponent Robotic System development

    2010, Information Sciences
    Citation Excerpt :

    One of the current challenges in this field is the optimal morphological design of the module in order to be able to produce some desired emergent properties through its integration in the MCRS [137]. Evolutionary algorithms, and other random search strategies, have been applied to this endeavour in the framework of evolutionary robotics [111,147]. The work in [175] discusses the limitations of metamorphic robots based on cubic modules.

  • Evolved neural networks based on cellular automata for sensory-motor controller

    2006, Neurocomputing
    Citation Excerpt :

    Smith et al. proposed the evolution of GasNet NN controllers for a robotic visual discrimination problem [20]. Santos et al. presented different aspects of the use of evolution for the successful generation of artificial NN controllers for real robot [19]. In their work, several parameters of an evolutionary/genetic algorithm (GA) and the way they influence the evolution of artificial neural network (ANN) behavioral controllers for real robots were contemplated.

  • Some thoughts on the use of sampled fitness functions for the multilevel Darwinist brain

    2004, Information Sciences
    Citation Excerpt :

    Obviously, the smaller the number of individuals in a population, the smaller the STM, and the less generations per data input cycle we run the evolutionary algorithm, the better in terms of resources. The problem regarding population size is that, as commented in [4], there is a lower limit we can use before running into premature convergence problems. Consequently, we will have to concentrate on the other two terms.

  • Evolutionary Architecture for Lifelong Learning and Real-Time Operation in Autonomous Robots

    2010, Evolving Intelligent Systems: Methodology and Applications
View all citing articles on Scopus
View full text