A Model-Based Reinforcement Learning Approach to Time-Optimal Control Problems

Abstract

Reinforcement Learning has achieved an exceptional performance in the last decade, yet its application to robotics and control remains a field for deeper investigation due to potential challenges. These include high-dimensional continuous state and action spaces, as well as complicated system dynamics and constraints in robotic settings. In this paper, we demonstrate a pioneering experiment in applying an existing model-based RL framework, PILCO, to the problem of time-optimal control. At first, the algorithm models the system dynamics with Gaussian Processes, successfully reducing the effect of model biases. Then, policy evaluation is done through iterated prediction with Gaussian posteriors and deterministic approximate inference. Finally, analytic gradients are used for policy improvement. A simulation and an experiment of an autonomous car completing a rest-to-rest linear locomotion is documented. Time-optimality and data efficiency of the task are shown in the simulation results, and learning under real-world circumstances is proved possible with our methodology.