Evolution of coordination as a metaphor for learning in multi-agent systems

Abstract

In societies of individually-motivated agents where the communication costs are prohibitive, there should be other mechanisms to allow them to coordinate. For instance, by choosing an equilibrium point. Such points have the property that, once all agents choose them, none can get higher utility from this joint decision. Hence, if this fact is common knowledge among agents, then an introspective reasoning process leads them to coordinate with low communication requirements. Game theory offers a mathematical formalism for modelling such interactions among agents. In classical game theory, agents or players are assumed to be always rational, which is a strong assumption regarding bounded-rational agents. Moreover, agents do not profit from results of past interactions. In the evolutionary approach to game theory on the other hand, agents do not need to have full knowledge about the rules of the game. Instead, they are involved in a learning process through active experimentation in which they may reach an equilibrium by playing repeatedly with neighbors. Such evolutionary approach is used here to model the interactions in a network of autonomous agents in the domain of control of traffic signals. The game is one of pure coordination with incomplete information. The dynamics of the game is determined by stochastic events which affect the traffic patterns forcing agents to adapt to them by altering their current strategies. Due to such perturbations, the equilibrium of the system eventually changes as well as the behavior of agents. If the game lasts long enough, agents can asymptotically learn how to re-coordinate their strategies and reach the global goal.

the author is supported by Conselho Nacional de Pesquisa Cientifica e Tecnológica — CNPq — Brasil.