Abstract
We present tools and guidelines for investigating Lipschitz continuity of the value functions in MDP’s, using the Hausdorff metric and the Kantorovich metric for measuring the influence of the constraint set and the transition law, respectively. The methods are explained by examples. Additional topics include an application to the the discretization algorithm of Bertsekas (1975).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Author information
Authors and Affiliations
Corresponding author
Additional information
Dedicated to Prof. Dr. Ulrich Rieder on the occasion of his 60th birthday
Part of the present paper was presented on June 15, 1995 at the Applied Probability Conference of the OR Society of America in Atlanta
Rights and permissions
About this article
Cite this article
Hinderer, K. Lipschitz Continuity of Value Functions in Markovian Decision Processes. Math Meth Oper Res 62, 3–22 (2005). https://doi.org/10.1007/s00186-005-0438-1
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/s00186-005-0438-1