Homo Egualis Reinforcement Learning Agents for Load Balancing

Abstract

Periodical policies were recently introduced as a solution for the coordination problem in games which assume competition between the players, and where the overall performance can only be as good as the performance of the poorest player. Instead of converging to just one Nash equilibrium, which may favor just one of the players, a periodical policy switches between periods in which all interesting Nash equilibria are played. As a result the players are able to equalize their pay-offs and a fair solution is build. Moreover players can learn this policy with a minimum on communication; now and then they send each other their performance. In this paper, periodical policies are investigated for use in real-life asynchronous games. More precisely we look at the problem of load balancing in a simple job scheduling game. The asynchronism of the problem is reflected in delayed pay-offs or reinforcements, probabilistic job creation and processor rates which follow an exponential distribution. We show that a group of homo egualis reinforcement learning agents can still find a periodical policy. When the jobs are small, homo egualis reinforcement learning agents find a good probability distribution over their action space to play the game without any communication.