Deep Q-Network Using Reward Distribution

Abstract

In this paper, we propose a Deep Q-Network using reward distribution. Deep Q-Network is based on the convolutional neural network which is a representative method of Deep Learning and the Q Learning which is a representative method of reinforcement learning. In the Deep Q-Network, when the game screen (observation) is given as an input to the convolutional neural network, the action value in Q Learning for each action is output. This method can realize learning that acquires a score equal to or higher than that of a human in plural games. The Q Learning learns using the greatest value in the next action, so a positive reward is propagated. However, since negative rewards can not be of greatest value, they are not propagated in learning. Therefore, by distributing negative rewards in the same way as Profit Sharing, the proposed method learn to not take wrong actions. Computer experiments were carried out, and it was confirmed that the proposed method can learn with almost the same speed and accuracy as the conventional Deep Q-Network. Moreover, by introducing reward distribution, we confirmed that learning can be performed so as not to acquire negative reward in the proposed method.