Abstract
We suggest a new algorithm for two-person zero-sum undiscounted stochastic games focusing on stationary strategies. Given a positive real \(\epsilon \), let us call a stochastic game \(\epsilon \)-ergodic, if its values from any two initial positions differ by at most \(\epsilon \). The proposed new algorithm outputs for every \(\epsilon >0\) in finite time either a pair of stationary strategies for the two players guaranteeing that the values from any initial positions are within an \(\epsilon \)-range, or identifies two initial positions \(u\) and \(v\) and corresponding stationary strategies for the players proving that the game values starting from \(u\) and \(v\) are at least \(\epsilon /24\) apart. In particular, the above result shows that if a stochastic game is \(\epsilon \)-ergodic, then there are stationary strategies for the players proving \(24\epsilon \)-ergodicity. This result strengthens and provides a constructive version of an existential result by Vrieze (1980) claiming that if a stochastic game is \(0\)-ergodic, then there are \(\epsilon \)-optimal stationary strategies for every \(\epsilon > 0\). The suggested algorithm extends the approach recently introduced for stochastic games with perfect information, and is based on the classical potential transformation technique that changes the range of local values at all positions without changing the normal form of the game.
Part of this research was done at the Mathematisches Forschungsinstitut Oberwolfach during a stay within the Research in Pairs Program from March 7 to March 20, 2010.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Boros, E., Elbassioni, K., Gurvich, V., Makino, K.: A pumping algorithm for ergodic stochastic mean payoff games with perfect information. In: Eisenbrand, F., Shepherd, F.B. (eds.) IPCO 2010. LNCS, vol. 6080, pp. 341–354. Springer, Heidelberg (2010)
Boros, E., Elbassioni, K., Gurvich, V., Makino, K.: On canonical forms for zero-sum stochastic mean payoff games. Dyn. Games Appl. 3(2), 128–161 (2013)
Blackwell, D., Ferguson, T.S.: The big match. Ann. Math. Statist. 39(1), 159–163 (1968)
Basu, S., Pollack, R., Roy, M.: On the combinatorial and algebraic complexity of quantifier elimination. J. ACM 43(6), 1002–1045 (1996). Preliminary version in FOCS 1994
Chatterjee, K., Ibsen-Jensen, R.: The complexity of ergodic mean-payoff games. In: Esparza, J., Fraigniaud, P., Husfeldt, T., Koutsoupias, E. (eds.) ICALP 2014, Part II. LNCS, vol. 8573, pp. 122–133. Springer, Heidelberg (2014)
Chatterjee, K., Majumdar, R., Henzinger, T.A.: Stochastic limit-average games are in exptime. Int. J. Game Theory 37, 219–234 (2008)
Federgruen, A.: Successive approximation methods in undiscounted stochastic games. Oper. Res. 1, 794–810 (1980)
Gallai, T.: Maximum-minimum Sätze über Graphen. Acta Mathematica Academiae Scientiarum Hungaricae 9, 395–434 (1958)
Gillette, D.: Stochastic games with zero stop probabilities. In: Dresher, M., Tucker, A.W., Wolfe, P. (eds) Contribution to the Theory of Games III, volume 39 of Annals of Mathematics Studies, pp. 179–187. Princeton University Press (1957)
Grigoriev, D., Vorobjov, N.: Solving systems of polynomial inequalities in subexponential time. J. Symb. Comput. 5(1/2), 37–64 (1988)
Hoffman, A.J., Karp, R.M.: On nonterminating stochastic games. Manag. Sci. Ser. A 12(5), 359–370 (1966)
Hansen, K.A., Koucky, M., Lauritzen, N., Miltersen, P.B., Tsigaridas. E.P.: Exact algorithms for solving stochastic games: extended abstract. In: Proceedings of the 43rd Annual ACM Symposium on Theory of Computing, STOC ’11, pp. 205–214. ACM, New York (2011)
Kemeny, J.G., Snell, J.L.: Finite Markov chains. Springer, New York (1963)
Miltersen, P.B.: Discounted stochastic games poorly approximate undiscounted ones, manuscript. Technical report (2011)
Mertens, J.F., Neyman, A.: Stochastic games. Int. J. Game Theory 10, 53–66 (1981)
Moulin, H.: Prolongement des jeux à deux joueurs de somme nulle. Bull. Soc. Math. France, Memoire, 45 (1976)
Renegar, J.: On the computational complexity and geometry of the first-order theory of the reals. J. Symb. Comput. 13(3), 255–352 (1992)
Raghavan, T.E.S., Filar, J.A.: Algorithms for stochastic games: a survey. Math. Methods Oper. Res. 35(6), 437–472 (1991)
Shapley, L.S.: Stochastic games. Proc. Nat. Acad. Sci. USA 39, 1095–1100 (1953)
Vrieze, O.J.: Stochastic games with finite state and action spaces. Ph.D. thesis, Centrum voor Wiskunde en Informatica, Amsterdam, The Netherlands (1980)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Boros, E., Elbassioni, K., Gurvich, V., Makino, K. (2014). A Potential Reduction Algorithm for Ergodic Two-Person Zero-Sum Limiting Average Payoff Stochastic Games. In: Zhang, Z., Wu, L., Xu, W., Du, DZ. (eds) Combinatorial Optimization and Applications. COCOA 2014. Lecture Notes in Computer Science(), vol 8881. Springer, Cham. https://doi.org/10.1007/978-3-319-12691-3_52
Download citation
DOI: https://doi.org/10.1007/978-3-319-12691-3_52
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-12690-6
Online ISBN: 978-3-319-12691-3
eBook Packages: Computer ScienceComputer Science (R0)