Andriy Burkov
ABSTRACT
It is known that the complexity of the reinforcement learning algorithms, such as Q-learning, may be exponential in the number of environment's states. It was shown, however, that the learning complexity for the goal-directed problems may be substantially reduced by initializing the Q-values with a "good" approximative function. In the multiagent case, there exists such a good approximation for a big class of problems, namely, for goal-directed stochastic games. These games, for example, can reflect coordination and common interest problems of cooperative robotics. The approximative function for these games is nothing but the relaxed, single-agent, problem solution, which can easily be found by each agent individually. In this article, we show that (1) an optimal single-agent solution is a "good" approximation for the goal-directed stochastic games with action-penalty representation and (b) the complexity is reduced when the learning is initialized with this approximative function, as compared to the uninformed case.
- O. Gies and B. Chaib-draa. Apprentissage de la coordination multiagent: une méthode basée sur le Q-learning par jeu adaptatif. Revue d'Intelligence Artificielle, 20(2-3):385--412, 2006.Google Scholar
Cross Ref
- S. Koenig and R. G. Simmons. The effect of representation and knowledge on goal-directed exploration with reinforcement-learning algorithms. Machine Learning, 22:227--250, 1996. Google ScholarDigital Library
- H. Young. The evolution of conventions. Econometrica, 61(1):57--84, 1993.Google ScholarCross Ref
- Reducing the complexity of multiagent reinforcement learning
Recommendations
A multiagent reinforcement learning algorithm using extended optimal response
AAMAS '02: Proceedings of the first international joint conference on Autonomous agents and multiagent systems: part 1Stochastic games provides a theoretical framework to multiagent reinforcement learning. Based on the framework, a multiagent reinforcement learning algorithm for zero-sum stochastic games was proposed by Littman and it was extended to general-sum games ...
Reward Shaping in Episodic Reinforcement Learning
AAMAS '17: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent SystemsRecent advancements in reinforcement learning confirm that reinforcement learning techniques can solve large scale problems leading to high quality autonomous decision making. It is a matter of time until we will see large scale applications of ...
Reinforcement social learning of coordination in cooperative multiagent systems
AAMAS '13: Proceedings of the 2013 international conference on Autonomous agents and multi-agent systemsCoordination in cooperative multiagent systems is an important problem and has received a lot of attention in multiagent learning literature. Most of previous works study the problem of how two (or more) players can coordinate on Pareto-optimal Nash ...
Comments