Culturing evolution strategies to support the exploration of novel environments by an intelligent robotic agent

Abstract

Recently, a lot of attention is paid to the use of evolutionary computing techniques for the development of adaptive robots. This paper presents the use of Cultural Algorithms with Evolution Strategies (ES) to produce optimal solutions to the ball following problem. This involves learning how to find and advance toward a ball. This is a basic skill needed to play robot soccer. Since it is possible that both the ball and the robot can be moving simultaneously this can be a difficult problem to solve. Four different solutions are implemented here. The first is the two member ES with comma strategy, (1,1)-ES. The second is the ES with plus strategy, (1+1)-ES. The third is a cultured version of the (1,1)-ES. The fourth is a cultured version of the (1+1)-ES. In cultural versions, the ES model is the population model for a Cultural Algorithm. The belief space contains generalizations about the individual's ancestral line, e.g. grandparents, in order to guide the modifications. Each of the four systems was tested by loading it into the RAM of a Khpera robot. Each system was then used to learn how to control the direction and speed of the two Khepera robot's wheels in order to direct it to a given ball. The real-time learning results are then compared. The results suggest that simple Evolution Strategies implemented here exhibit a satisfactory level of real-time learning in forwarding and pushing the ball. The cultured version exhibits an improvement in time over this performance as well.