Abstract
In two-player games on graphs, the players move a token through a graph to produce an infinite path, which determines the qualitative winner or quantitative payoff of the game. In bidding games, in each turn, we hold an auction between the two players to determine which player moves the token. Bidding games have largely been studied with concrete bidding mechanisms that are variants of a first-price auction: in each turn both players simultaneously submit bids, the higher bidder moves the token, and pays his bid to the lower bidder in Richman bidding, to the bank in poorman bidding, and in taxman bidding, the bid is split between the other player and the bank according to a predefined constant factor. Bidding games are deterministic games. They have an intriguing connection with a fragment of stochastic games called random-turn games. We study, for the first time, a combination of bidding games with probabilistic behavior; namely, we study bidding games that are played on Markov decision processes, where the players bid for the right to choose the next action, which determines the probability distribution according to which the next vertex is chosen. We study parity and mean-payoff bidding games on MDPs and extend results from the deterministic bidding setting to the probabilistic one.
This research was supported in part by the Austrian Science Fund (FWF) under grants S11402-N23 (RiSE/SHiNE), Z211-N23 (Wittgenstein Award), and M 2369-N33 (Meitner fellowship), and the Czech Science Foundation grant no. GJ19-15134Y.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
When the initial ratio is exactly \(\texttt {Thresh} (v)\), the winner depends on the mechanism with which ties are broken. Our results do not depend on a specific tie-breaking mechanism.Tie-breaking mechanisms are particularly important in discrete-bidding games [1].
References
Aghajohari, M., Avni, G., Henzinger, T.A.: Determinacy in discrete-bidding infinite-duration games. In: Proceedings 30th CONCUR (2019)
Apt, K.R., Grädel, E.: Lectures in Game Theory for Computer Scientists. Cambridge University Press, Cambridge (2011)
Avni, G., Henzinger, T.A., Chonev, V.: Infinite-duration bidding games. J. ACM 66(4), 31:1–31:29 (2019)
Avni, G., Henzinger, T.A., Ibsen-Jensen, R.: Infinite-duration poorman-bidding games. In: Christodoulou, G., Harks, T. (eds.) WINE 2018. LNCS, vol. 11316, pp. 21–36. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04612-5_2
Avni, G., Henzinger, T.A., Žikelić, Đ.: Bidding mechanisms in graph games. In: Proceedings of the 44th MFCS (2019)
Chatterjee, K.: Robustness of structurally equivalent concurrent parity games. In: Birkedal, L. (ed.) FoSSaCS 2012. LNCS, vol. 7213, pp. 270–285. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28729-9_18
Condon, A.: The complexity of stochastic games. Inf. Comput. 96(2), 203–224 (1992)
Howard, A.R.: Dynamic Programming and Markov Processes. MIT Press, Cambridge (1960)
Lazarus, A.J., Loeb, D.E., Propp, J.G., Stromquist, W.R., Ullman, D.H.: Combinatorial games under auction play. Games Econ. Behav. 27(2), 229–264 (1999)
Lazarus, A.J., Loeb, D.E., Propp, J.G., Ullman, D.: Richman games. Games No Chance 29, 439–449 (1996)
Peres, Y., Schramm, O., Sheffield, S., Wilson, D.B.: Tug-of-war and the infinity laplacian. J. Amer. Math. Soc. 22, 167–210 (2009)
Pnueli, A., Rosner, R.: On the synthesis of a reactive module. In: Proceedings of the 16th POPL, pp. 179–190 (1989)
Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York (2005)
Rabin, M.O.: Decidability of second order theories and automata on infinite trees. Trans. AMS 141, 1–35 (1969)
Solan, E.: Continuity of the value of competitive Markov decision processes. J. Theor. Probab. 16, 831–845 (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Avni, G., Henzinger, T.A., Ibsen-Jensen, R., Novotný, P. (2019). Bidding Games on Markov Decision Processes. In: Filiot, E., Jungers, R., Potapov, I. (eds) Reachability Problems. RP 2019. Lecture Notes in Computer Science(), vol 11674. Springer, Cham. https://doi.org/10.1007/978-3-030-30806-3_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-30806-3_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30805-6
Online ISBN: 978-3-030-30806-3
eBook Packages: Computer ScienceComputer Science (R0)