Abstract.
This paper presents three conditions. Each of them guarantees the uniqueness of optimal policies of discounted Markov decision processes. The conditions presented here impose hypotheses specifically on the state space X, the action space A, the admissible action sets A(x),x∈X, the transition probability Q, and on the cost function c. Two of these conditions require mainly convexity assumptions, but the third one does not need this kind of assumptions. However, it needs certain stochastic order relations in Q, and the cost function c to reach its minimum with respect to the actions, just in one action. We illustrate the conditions with several examples including, in particular, discrete models, the linear regulator problem, and also a model of an inventory control system.
Similar content being viewed by others
Author information
Authors and Affiliations
Corresponding author
Additional information
Manuscript received: May 2003 / Final version received: January 2004
Rights and permissions
About this article
Cite this article
Cruz-Suárez, D., Montes-de-Oca, R. & Salem-Silva, F. Conditions for the uniqueness of optimal policies of discounted Markov decision processes. Math Meth Oper Res 60, 415–436 (2004). https://doi.org/10.1007/s001860400372
Issue Date:
DOI: https://doi.org/10.1007/s001860400372