Skip to main content
SpringerLink
Log in

Elementary Subpaths in Discounted Stochastic Games

  • Published:
Dynamic Games and Applications Aims and scope Submit manuscript

Abstract

This paper examines the subgame-perfect equilibria in discounted stochastic games with finite state and action spaces. The fixed-point characterization of equilibria is generalized to unobservable mixed strategies. It is also shown that the pure-strategy equilibria consist of elementary subpaths, which are repeating fragments that give the acceptable action plans in the game. The developed methodology offers a novel way of computing and analyzing equilibrium strategies that need not be stationary nor Markovian.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. See [28, 34, 51] for the use of punishment strategies in stochastic games.

References

  1. Abreu D (1986) Extremal equilibria of oligopolistic supergames. J Econ Theory 39(2):191–225

    Article  MathSciNet  MATH  Google Scholar 

  2. Abreu D (1988) On the theory of infinitely repeated games with discounting. Econometrica 56(2):383–396

    Article  MathSciNet  MATH  Google Scholar 

  3. Abreu D, Pearce D, Stacchetti E (1986) Optimal cartel equilibria with imperfect monitoring. J Econ Theory 39(1):251–269

    Article  MathSciNet  MATH  Google Scholar 

  4. Abreu D, Pearce D, Stacchetti E (1990) Toward a theory of discounted repeated games with imperfect monitoring. Econometrica 58(5):1041–1063

    Article  MathSciNet  MATH  Google Scholar 

  5. Amir R (2003) Stochastic games in economics and related fields: an overview. In: Neyman A, Sorin S (eds) Stochastic games and applications, NATO Science, vol 570. Kluwer, Dordrecht, pp 455–470

    Chapter  Google Scholar 

  6. Atkeson A (1991) International lending with moral hazard and risk of repudiation. Econometrica 59(4):1069–1089

    Article  MathSciNet  MATH  Google Scholar 

  7. Aumann RJ (1981) Survey of repeated games. Essays in game theory and mathematical economics in Honor of Oskar Morgenstern. Bibliographisches Insitut, Mannheim, pp 11–42

    Google Scholar 

  8. Balbus L, Reffett K, Wozny L (2013) Markov stationary equilibria in stochastic supermodular games with imperfect private and public information. Dyn Games Appl 3:187–206

    Article  MathSciNet  MATH  Google Scholar 

  9. Balbus L, Reffett K, Wozny L (2014) A constructive study of Markov equilibria in stochastic games with strategic complementarities. J Econ Theory 150:815–840

    Article  MathSciNet  MATH  Google Scholar 

  10. Berg K (2013) Extremal pure strategies and monotonicity in repeated games. Working paper

  11. Berg K, Kärki M (2014) An algorithm for finding the minimal pure-strategy subgame-perfect equilibrium payoffs in repeated games. Working paper

  12. Berg K, Kitti M (2012) Equilibrium paths in discounted supergames. Working paper

  13. Berg K, Kitti M (2013) Computing equilibria in discounted \(2\times 2\) supergames. Comput Econ 41(1):71–88

    Article  Google Scholar 

  14. Berg K, Kitti M (2014) Fractal geometry of equilibrium payoffs in discounted supergames. Fractals 22(4). doi:10.1142/S0218348X14500169

  15. Berg K, Schoenmakers G (2014) Construction of randomized subgame-perfect equilibria in repeated games. Working paper

  16. Blackwell D, Ferguson TS (1968) The big match. Ann Math Stat 39(1):159–163

    Article  MathSciNet  MATH  Google Scholar 

  17. Borkovsky RN, Doraszelski U, Kryukov Y (2010) A user’s guide to solving dynamic stochastic games using the homotopy method. Oper Res 58(2):1116–1132

    Article  MathSciNet  MATH  Google Scholar 

  18. Burkov A, Chaib-Draa B (2010) An approximate subgame-perfect equilibrium computation technique for repeated games. In: Proceedings of the twenty-fourth AAAI conference on artificial intelligence, pp 729–736

  19. Cole HL, Kocherlakota N (2001) Dynamic games with hidden actions and hidden states. J Econ Theory 98:114–126

    Article  MathSciNet  MATH  Google Scholar 

  20. Cronshaw MB (1997) Algorithms for finding repeated game equilibria. Comput Econ 10:139–168

    Article  MATH  Google Scholar 

  21. Doraszelski U, Pakes A (2007) A framework for applied dynamic analysis in IO. In: Armstrong M, Porter R (eds) Handbook of industrial organization 3, pp 1887–1966, North-Holland

  22. Doraszelski U, Escobar JF (2012) Restricted feedback in long term relationships. J Econ Theory 147(1):142–161

    Article  MathSciNet  MATH  Google Scholar 

  23. Ericson R, Pakes A (1995) Markov-perfect industry dynamics: a framework for empirical work. Rev Econ Stud 62(1):53–82

    Article  MATH  Google Scholar 

  24. Feng Z, Miao J, Peralta-Alva A, Santos MS (2014) Numerical simulation of nonoptimal dynamic equilibrium models. Int Econ Rev 55(1):83–110

    Article  MathSciNet  MATH  Google Scholar 

  25. Filar J, Vrieze K (1997) Competitive markov decision processes. Springer, Berlin

    MATH  Google Scholar 

  26. Fink AM (1964) Equilibrium in stochastic n-person games. J Sci Hiroshima Univ Univ Ser A-I 28:89–93

    MathSciNet  MATH  Google Scholar 

  27. Flesch J, Thuijsman F, Vrieze OJ (1997) Cyclic Markov equilibria in stochastic games. Int J Game Theory 26:303–314

    Article  MathSciNet  MATH  Google Scholar 

  28. Flesch J, Thuijsman F, Vrieze OJ (2000) Almost stationary \(\epsilon \)-equilibria in zero-sum stochastic games. J Optim Theory Appl 105(2):371–389

    Article  MathSciNet  MATH  Google Scholar 

  29. Fudenberg D, Yamamoto Y (2011) The folk theorem for irreducible stochastic games with imperfect public monitoring. J Econ Theory 146:1664–1683

    Article  MathSciNet  MATH  Google Scholar 

  30. Govindan S, Wilson R (2009) Global Newton method for stochastic games. J Econ Theory 144:414–421

    Article  MathSciNet  MATH  Google Scholar 

  31. Herings JJ, Peeters R (2004) Stationary equilibria in stochastic games: structure, selection and computation. J Econ Theory 118(1):32–60

    MathSciNet  MATH  Google Scholar 

  32. Hörner J, Sugaya T, Takahashi S, Vieille N (2011) Recursive methods in discounted stochastic games: an algorithm for \(\delta \rightarrow 1\) and a folk theorem. Econometrica 79(4):1277–1318

    Article  MathSciNet  MATH  Google Scholar 

  33. Judd K, Yeltekin Ş, Conklin J (2003) Computing supergame equilibria. Econometrica 71:1239–1254

    Article  MathSciNet  MATH  Google Scholar 

  34. Kitti M (2013a) Conditional Markov equilibria in discounted dynamic games. Math Methods Oper Res 78:77–100

    Article  MathSciNet  MATH  Google Scholar 

  35. Kitti M (2013) Subgame perfect equilibria in discounted stochastic games. Working paper

  36. Levy Y (2013) Discounted stochastic games with no stationary Nash equilibrium: two examples. Econometrica 81(5):1973–2007

    Article  MathSciNet  MATH  Google Scholar 

  37. Levy Y, McLennan A (2015) Corrigendum to: discounted stochastic games with no stationary Nash equilibrium: two examples. Econometrica (to appear)

  38. MacDermed L, Narayan KS, Isbell CL, Weiss L (2011) Quick polytope approximation of all correlated equilibria in stochastic games. In: Proceedings of the twenty-fifth AAAI conference on artificial intelligence, pp 707–712

  39. Mailath GJ, Samuelson L (2006) Repeated games and reputations: long-run relationships. Oxford University Press, Oxford

    Book  Google Scholar 

  40. Mertens J-F, Parthasarathy TES (1987) Equilibria for discounted stochastic games. CORE discussion paper 8750, Universite Catholique de Louvain, Belgium

  41. Mertens J-F, Parthasarathy TES (2003) Equilibria for discounted stochastic games. In: Neyman A, Sorin S (eds) Stochastic games and applications. NATO science series C, mathematical and physical sciences, vol 570. Kluwer, Dordrecht, pp 131–172

    Chapter  Google Scholar 

  42. Murray C, Gordon GJ (2007) Multi-robot negotiation: approximating the set of subgame perfect equilibria in general-sum stochastic games. Adv Neural Inf Process Syst 19:1001–1008

    Google Scholar 

  43. Pakes A, Ericson R (1998) Empirical implications of alternative models of firm dynamics. J Econ Theory 79:1–45

    Article  MATH  Google Scholar 

  44. Phelan C, Stacchetti E (2001) Sequential equilibria in a Ramsey tax model. Econometrica 69(6):1491–1518

    Article  MathSciNet  MATH  Google Scholar 

  45. Prasad H L, Prashanth LA, Shalabh B (2014) Algorithms for Nash equilibria in general-sum stochastic games. Working paper

  46. Sen A, Shanno DF (2008) Optimization and dynamical systems algorithms for finding equilibria of stochastic games. Opt Method Softw 23(6):975–993

    Article  MathSciNet  MATH  Google Scholar 

  47. Shapley LS (1953) Stochastic games. Proc Natl Acad Sci USA 39:1095–1100

    Article  MathSciNet  MATH  Google Scholar 

  48. Sleet C, Yeltekin S (2015) On the computation of value correspondences for dynamic games. Dyn Games Appl. doi:10.1007/s13235-015-0139-1

  49. Solan E, Vieille N (2001) Quitting games. Math Oper Res 26(2):265–285

    Article  MathSciNet  MATH  Google Scholar 

  50. Takahashi M (1964) Equilibrium points of stochastic non-cooperative n-person games. J Sci Hiroshima Univ Ser A-I 28:95–99

    MathSciNet  MATH  Google Scholar 

  51. Thuijsman F, Vrieze OJ (1997) The power of threats in stochastic games. In: Bardi et al. (eds) Stochastic games and numerical methods for dynamic games, Birkhauser, pp 339–353

  52. Yamamoto Y (2015) Stochastic games with hidden states. Working paper

  53. Yeltekin S, Cai Y, Judd KL (2015) Computing equilibria of dynamic games. Working paper

Download references

Acknowledgments

The author is grateful for the reviewers’ comments and suggestions for improvements. The author acknowledges funding from Emil Aaltosen Säätiö through Post doc -pooli.

Author information

Authors and Affiliations

  1. Department of Mathematics and Systems Analysis, Aalto University School of Science, P.O. Box 11100, 00076, Aalto, Finland

    Kimmo Berg

Authors
  1. Kimmo Berg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kimmo Berg.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Berg, K. Elementary Subpaths in Discounted Stochastic Games. Dyn Games Appl 6, 304–323 (2016). https://doi.org/10.1007/s13235-015-0151-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13235-015-0151-5

Keywords

Search

Navigation