Skip to main content

Advertisement

Log in

The finiteness of the reward function and the optimal value function in Markov decision processes

  • Published:
Mathematical Methods of Operations Research Aims and scope Submit manuscript

Abstract.

This paper studies the discrete time Markov decision processes (MDP) with expected discounted total reward, where the state space is countable, the action space is measurable, the reward function is extended real-valued, and the discount rate may be any real number. Two conditions (GC) and (C) are presented, which are weaker than that presented in literature. By eliminating some worst actions, the state space S can be partitioned into sets S , S −∞, S 0, on which the optimal value function equals +∞ , −∞ or is finite, respectively. Furthermore, the validity of the optimality equation is shown when its right hand side is well defined, especially, when it is restricted to the subset S 0. The reward function r (i, a) is finite and bounded above in a for each iS 0. Finally, some sufficient conditions for (GC) and (C) are given.

This is a preview of subscription content, log in via an institution to check access.

Access this article

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Similar content being viewed by others

Author information

Authors and Affiliations

Authors

Additional information

Manuscript received: Februry 1998/final version received: October 1998

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hu, Q., Xu, C. The finiteness of the reward function and the optimal value function in Markov decision processes. Mathematical Methods of OR 49, 255–266 (1999). https://doi.org/10.1007/PL00020916

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/PL00020916

Navigation