Loading [a11y]/accessibility-menu.js
Stochastic Subgradient Methods for Dynamic Programming in Continuous State and Action Spaces | IEEE Conference Publication | IEEE Xplore

Stochastic Subgradient Methods for Dynamic Programming in Continuous State and Action Spaces


Abstract:

In this paper, we propose a numerical method for dynamic programming in continuous state and action spaces. We first approximate the Bellman operator by using a convex op...Show More

Abstract:

In this paper, we propose a numerical method for dynamic programming in continuous state and action spaces. We first approximate the Bellman operator by using a convex optimization problem, which has many constraints. This convex program is then solved using stochastic subgradient descent. To avoid the full projection onto the high-dimensional feasible set, we develop a novel algorithm that samples, in a coordinated fashion, a mini-batch for a subgradient and another for projection. We show several salient properties of this algorithm, including convergence, and a reduction in the feasibility error and in the variance of the stochastic subgradient.
Date of Conference: 11-13 December 2019
Date Added to IEEE Xplore: 12 March 2020
ISBN Information:

ISSN Information:

Conference Location: Nice, France

References

References is not available for this document.