Learning by linear anticipation in multi-agent systems

Abstract

A linearly anticipatory agent architecture for learning in multi-agent systems is presented. It integrates low-level reaction with high-level deliberation by embedding an ordinary reactive system based on situation-action rules, called the Reactor, in an anticipatory agent forming a layered hybrid architecture. By treating all agents in the domain (itself included) as being reactive, this approach reduces the amount of search needed while at the same time requiring only a small amount of heuristic domain knowledge. Instead it relies on a linear anticipation mechanism, carried out by the Anticipator, to learn new reactive behaviors. The Anticipator uses a world model (in which all agents are represented only by their Reactor) to make a sequence of one-step predictions. After each step it checks whether an undesired state has been reached. If this is the case it will adapt the actual Reactor in order to avoid this state in the future. Results from simulations on learning reactive rules for cooperation and coordination of teams of agents indicate that the behavior of this type of agents is superior to that of the corresponding reactive agents. Also some promising results from simulations of competing self-interested agents are presented.