Effects of Chaotic Exploration on Reinforcement Maze Learning

Abstract

In reinforcement learning, it is necessary to introduce a process of trial and error called an exploration. As a generator for exploration, it seems to be familiar to use the uniform pseudorandom number generator. However, it is known that chaotic source also provides a random-like sequence as like as stochastic source. In this research, we propose an application of the random-like feature of deterministic chaos for a generator of the exploration. As a result, we find that the deterministic chaotic generator for the exploration based on the logistic map gives better performances than the stochastic random exploration generator in a nonstationary shortcut maze problem. In order to understand why the exploration generator based on the logistic map shows the better result, we investigate the learning structures obtained from the two exploration generators.