Abstract:
In this paper, we propose a numerical method for dynamic programming in continuous state and action spaces. We first approximate the Bellman operator by using a convex op...Show MoreMetadata
Abstract:
In this paper, we propose a numerical method for dynamic programming in continuous state and action spaces. We first approximate the Bellman operator by using a convex optimization problem, which has many constraints. This convex program is then solved using stochastic subgradient descent. To avoid the full projection onto the high-dimensional feasible set, we develop a novel algorithm that samples, in a coordinated fashion, a mini-batch for a subgradient and another for projection. We show several salient properties of this algorithm, including convergence, and a reduction in the feasibility error and in the variance of the stochastic subgradient.
Published in: 2019 IEEE 58th Conference on Decision and Control (CDC)
Date of Conference: 11-13 December 2019
Date Added to IEEE Xplore: 12 March 2020
ISBN Information: