Abstract
We present two algorithms to find the component-wise upper and lower bounds of the steady-state distribution of an ergodic Markov chain. whose transition matrix\(\mathbf{M}\) is entry-wise larger than matrix \(\mathbf{L}\). The algorithms are faster than Muntz’s approach. They are based on the polyhedral theory developed by Courtois and Semal and on a new iterative algorithm which gives bounds of the steady-state distribution at each iteration.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Buchholz, P.: An improved method for bounding stationary measures of finite Markov processes. Perform. Eval. 62, 349–365 (2005)
Buchholz, P.: Bounding reward measures of markov models using the Markov decision processes. Numer. Linear Algebra Appl. 18(6), 919–930 (2011)
Busic, A., Fourneau, J.-M.: Iterative component-wise bounds for the steady-state distribution of a Markov chain. Numer. Linear Algebra Appl. 18(6), 1031–1049 (2011)
Courtois, P.-J., Semal, P.: Bounds for the positive eigenvectors of nonnegative matrices and for their approximations by decomposition. J. ACM 31(4), 804–825 (1984)
Fourneau, J.-M., Le Coz, M., Pekergin, N., Quessette, F.: An open tool to compute stochastic bounds on steady-state distributions and rewards. In: 11th International Workshop on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS 2003). IEEE Computer Society, Orlando, FL (2003)
Fourneau, J.-M., Quessette, F.: Some improvements for the computation of the steady-state distribution of a Markov chain by monotone sequences of vectors. In: ASMTA, volume 7314 of Lecture Notes in Computer Science, pp. 178–192. Springer (2012)
Franceschinis, G., Muntz, R.R.: Bounds for quasi-lumpable Markov chains. In: Performance ’93, pp. 223–243. Elsevier Science Publishers B. V. (1994)
Grassman, W., Taksar, M., Heyman, D.: Regenerative analysis and steady state distributions for Markov chains. Oper. Res. 33(5), 1107–1116 (1985)
Stewart, W.: Introduction to the Numerical Solution of Markov Chains. Princeton University Press, New Jersey (1995)
Acknowledgments
This work was partially supported by a grant from CNRS GdR RO 2011.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag London
About this paper
Cite this paper
Salaht, F.A., Fourneau, J.M., Pekergin, N. (2013). Computing Entry-Wise Bounds of the Steady-State Distribution of a Set of Markov Chains. In: Gelenbe, E., Lent, R. (eds) Computer and Information Sciences III. Springer, London. https://doi.org/10.1007/978-1-4471-4594-3_12
Download citation
DOI: https://doi.org/10.1007/978-1-4471-4594-3_12
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-4471-4593-6
Online ISBN: 978-1-4471-4594-3
eBook Packages: EngineeringEngineering (R0)