An approximation approach to ergodic semi-Markov control processes

Abstract.

We consider semi-Markov control models (SMCMs) with a Borel state space satisfying certain stochastic stability assumptions on the transition structure which imply the so-called V-uniform geometric ergodicity of the state process. We deal with a class of ε-perturbations of transition probability functions of the original model. First, we determine the rate of convergence of the optimal expected costs in in perturbed models to the optimal expected cost in the orginal SMCM. Next, we present a new algorithm for finding the solution to the average cost optimality equation (ACOE). The algorithm makes use of a sequence of solutions to the ACOE for the perturbed models, which can be found by a simple iterative procedure.