Learning fuzzy classifier systems for multi-agent coordination

Abstract

We present ELF, a learning fuzzy classifier system (LFCS), and its application to the field of Learning Autonomous Agents. In particular, we will show how this kind of Reinforcement Learning systems can be successfully applied to learn both behaviors and their coordination for Autonomous Agents. We will discuss the importance of knowledge representation approach based on fuzzy sets to reduce the search space without losing the required precision. Moreover, we will show how we have applied ELF to learn the distributed coordination among agents which can exchange information with each other. The experimental validation has been done on software agents interacting in a real-time task.

Introduction

In the last decade, we have worked on the definition of learning classifier systems (LCS) [1], [2], [3] whose rules (or classifiers) are implemented as fuzzy rules (learning fuzzy classifier systems (LFCS)) [4], [5], [6]. This makes it possible to match real-valued input preserving most of its informative content, whereas traditional LCS lose part of this information, since their classifiers match real-valued input with rules whose antecedent is composed of interval-valued variables. Moreover, at each activation cycle, an LFCS composes the output of different fuzzy rules to obtain a real-valued output, while in traditional LCS, a single rule is selected, proposing an output from a small set of different values. In Section 2 we briefly introduce the context where evolutionary learning of fuzzy rules (ELF) was developed, then describing the principles on which it is based. In other papers we have discussed the application of ELF to real and simulated robotics agents [7], [8], [9], and some of the various methodological aspects related to a whole class of LFCS [6], [10]. Here, we focus on a challenging task, such as simulated robotics soccer, in the distributed Robocup environment [11], which can be summarized as follows. We have a set of autonomous agents, i.e., entities governed by software programs, able to perform a task autonomously in an environment perceived by a set of sensors. Behaviors shown by each agent are obtained by the interaction of relatively simple and independent behavior modules [12]. One of the problems of this behavior-based approach is the composition of concurrent behaviors, which is usually designed by hand. Elsewhere we have shown as ELF can learn single behavior modules [7], and the selection/composition of behavior modules [8], [9]. In Section 3, we propose another, more complete, solution to this problem, which also takes into account the role of interaction with other agents in the optimization of the behavior blend.

An agent may have to interact with other agents to perform its task. An important aspect of its global behavior concerns the modality of this interaction, and this is also an issue so complex that learning may play a key role in facing it. In this paper we will show how ELF can be applied in a simulated robotics soccer context, where agents can communicate to improve their limited perception of an environment where they have to interact with agents which may contrast them. This happens in a real-time, dynamic environment (provided by Robocup [11]), with hard technological constraints, which will be described in Section 3. In Section 4, we present experimental results showing the effectiveness of ELF in learning a distributed policy for the interacting agents.