Comparing Distributed Reinforcement Learning Approaches to Learn Agent Coordination

Abstract

This work compares the performance of the Ant-ViBRA system to approaches based on Distributed Q-learning and Q-learning, when they are applied to learn coordination among agent actions in a Multi Agent System. Ant-ViBRA is a modified version ofa Swarm Intelligence Algorithm called the Ant Colony System algorithm (ACS), which combines a Reinforcement Learning (RL) approach with Heuristic Search. Ant-ViBRA uses a priori domain knowledge to decompose the domain task into subtasks and to define the relationship between actions and states based on interactions among subtasks. In this way, Ant-ViBRA is able to cope with planning when several agents are involved in a combinatorial optimization problem where interleaved execution is needed. The domain in which the comparison is made is that ofa manipulator performing visually-guided pick-and-place tasks in an assembly cell. The experiments carried out are encouraging, showing that Ant- ViBRA presents better results than the Distributed Q-learning and the Q-learning algorithms.