Abstract
Conditions are provided to derive error bounds on the effect of truncations and perturbations in Markov decision problems. Both the average and finite horizon case are studied. As an application, an explicit error bound is obtained for a truncation of a Jacksonian queueing network with overflow control.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
K. Hinderer, On approximate solutions of finite-stage dynamic programs,Dynamic Programming and its Applications, ed. M.L. Puterman (Academic Press, New York, 1978).
M. Kolonko, Bounds for the regret loss in dynamic programming, under adaptive control, Zeit. O.R. 27 (1983) 17–37.
M. Kurano, Average-optimal adaptive policies in semi-Markov decision processes including an unknown parameter, J. Oper. Res. Soc. Japan 28 (1985) 252–266.
H.J. Langen, Convergence of dynamic programming models, Math. Oper. Res. 6 (1981) 493–512.
C.D. Meyer Jr., The condition of a finite Markov chain and perturbation bounds for the limiting probabilities, SIAM J. Alg. Disc. Math. 1 (1980) 273–283.
T.J. Ott and K.R. Krishnan, State dependent routing of telephone traffic and the use of separable routing schemes,Proc. 11th Int. Teletraffic Congres, Tokyo, Japan (1985).
S.M. Ross,Applied Probability Models with Optimization Application (Holden-Day, San Francisco, 1970).
T.A. Sarymsakov, Sur les chaines de Markoff à une infinité dénombrable d'états possibles, Doklady Akad. Sci. U.S.S.R. 48 (1975) 159–161.
P.J. Schweitzer, Perturbation theory and finite Markov chains, J. Appl. Prob. 5 (1968) 401–413.
E. Seneta, Finite approximations to infinite non-negative matrices, Proc. Cambridge Phil. Soc. 63 (1967) 983–992.
E. Seneta, The principles of truncations in applied probability, Comm. Math. Univ. Carolina 9 (1968) 533–539.
E. Seneta,Non-Negative Matrices and Markov Chains (Springer, 1980).
H.C. Tijms,Stochastic Modelling and Analysis. A Computational Approach (Wiley, New York, 1986).
E.A. Van Doorn, On the overflow process from a finite Markovian queue, Performance Eval. 4 (1984) 233–240.
N.M. Van Dijk, Perturbation theory for unbounded Markov reward processes with applications to queueing, Adv. Appl. Prob. 20 (1988) 99–111.
N.M. Van Dijk, A proof of simple insensitive bounds for a pure overflow system, J. Appl. Prob. 26 (1989) 113–120.
N.M. Van Dijk, Truncation of Markov chains with applications to queueing, Research Report 53, Free University, Amsterdam (1989).
N.M. Van Dijk, and M.L. Puterman, Perturbation theory for Markov reward processes with applications to queueing systems, Adv. Appl. Prob. 20 (1988) 79–98.
N.M. Van Dijk, P. Tsoucas and J. Walrand, Simple bounds and monotonicity of the call congestion of finite multiserver delay systems, Prob. Eng. Inf. Sci. 2 (1988) 129–138.
W. Whitt, Approximations of dynamic programs I, Math. Oper. Res. 3 (1978) 231–243.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Van Dijk, N.M. On truncations and perturbations of Markov decision problems with an application to queueing network overflow control. Ann Oper Res 29, 515–535 (1991). https://doi.org/10.1007/BF02283612
Issue Date:
DOI: https://doi.org/10.1007/BF02283612