Skip to main content
SpringerLink
Log in

A Strategic Dynamic Programming Method for Studying Short-Memory Equilibria of Stochastic Games with Uncountable Number of States

  • Published:
Dynamic Games and Applications Aims and scope Submit manuscript

Abstract

We study a class of infinite horizon stochastic games with uncountable number of states. We first characterize the set of all (nonstationary) short-term (Markovian) equilibrium values by developing a new (Abreu et al. in Econometrica 58(5):1041–1063, 1990)-type procedure operating in function spaces. This (among others) proves Markov perfect Nash equilibrium (MPNE) existence. Moreover, we present techniques of MPNE value set approximation by a sequence of sets of discretized functions iterated on our approximated APS-type operator. This method is new and has some advantages as compared to Judd et al. (Econometrica 71(4):1239–1254, 2003), Feng et al. (Int Econ Rev 55(1):83–110, 2014), or Sleet and Yeltekin (Dyn Games Appl doi:10.1007/s13235-015-0139-1, 2015). We show applications of our approach to hyperbolic discounting games and dynamic games with strategic complementarities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. As analyzed in [8, 10, 31, 32, 46] or [15], for example.

  2. As analyzed in [5, 26, 45] or [14], for example.

  3. See also [41] or [42], pages 397–398.

  4. It bears mentioning, we focus on short-memory Markovian equilibrium because this class of equilibrium has been the focus of a great deal of applied work. We should also mention very interesting papers by Cole and Kocherlakota [24] and Doraszelski and Escobar [27] that also pursue a similar idea of trying to develop MP/APS-type procedure in function spaces for Markovian equilibrium (i.e., methods where continuation structures are parameterized by functions) but for finite/countable number of states. See also [18] for a related argument used to prove existence of equilibrium in a bequest game.

  5. In our accompanied papers (see [11, 14, 15]), we propose an alternative direct method for stationary Markov equilibrium existence and computation. We view results of both, direct and indirect, methods as complementary and discuss them in the paper.

  6. It bears mentioning that for dynamic games with more restrictive shocks spaces (e.g., discrete or countable), MP/APS procedure has been used extensively in economics in recent years: see e.g., [18] for altruistic economies, [7, 37, 48, 49] for policy games or FMPS for recursive competitive equilibrium of a dynamic economy.

  7. For example, see Phelan and Stacchetti, who discuss such possibility in function spaces.

  8. See also [22], who use APS technique to analyze equilibria of a n-player quasi-hyperbolic discounting game with imperfect monitoring.

  9. See also [4] or [44] for related assumptions on the transition probability.

  10. The assumption we impose here are very similar as those in the work of Amir [6] for \(S\subset \mathbb {R}\).

  11. That is \(\int \limits _Sf(s')Q(ds'|\cdot ,\cdot )\) is continuous, whenever f is continuous and bounded.

  12. That is, player i uses strategy \(\tilde{\gamma }\) up to period T and \(\gamma \) after that. Other players use \(\gamma \).

  13. Also Cronshaw [25] proposes a Newton method for equilibrium value set approximation but cannot prove that his procedure converges to the greatest fixed point of our interest.

  14. To avoid technical difficulties with defining the values of v at \(\bar{S}\), we should extend the domain \([0,\bar{S}]\) to some \([0,\bar{S}']\) with \(\bar{S}'>\bar{S}\). See [29] for a formal argument.

References

  1. Abreu D, Pearce D, Stacchetti E (1986) Optimal cartel equilibria with imperfect monitoring. J Econ Theory 39(1):251–269

    Article  MathSciNet  MATH  Google Scholar 

  2. Abreu D, Pearce D, Stacchetti E (1990) Toward a theory of discounted repeated games with imperfect monitoring. Econometrica 58(5):1041–1063

    Article  MathSciNet  MATH  Google Scholar 

  3. Aliprantis CD, Border KC (2005) Infinite dimensional analysis. A Hitchhiker’s guide. Springer, Heilbelberg

    MATH  Google Scholar 

  4. Amir R (1996) Continuous stochastic games of capital accumulation with convex transitions. Games Econ Behav 15(2):111–131

    Article  MathSciNet  MATH  Google Scholar 

  5. Amir R (2002) Complementarity and diagonal dominance in discounted stochastic games. Ann Oper Res 114(1–4):39–56

    Article  MathSciNet  MATH  Google Scholar 

  6. Amir R (2010) Discounted supermodular stochastic games: theory and applications. Mimeo

  7. Atkeson A (1991) International lending with moral hazard and risk of repudiation. Econometrica 59(4):1069–89

    Article  MathSciNet  MATH  Google Scholar 

  8. Balbus Ł, Jaśkiewicz A, Nowak AS (2014) Robust Markov perfect equilibria in a dynamic choice model with quasi-hyperbolic discounting. In: Haunschmied J, Veliov V, Wrzaczek S (eds) Dynamic games in economics. Dynamic modeling and econometrics in economics and finance, vol 16. Springer, Berlin, pp 1–22

    Google Scholar 

  9. Balbus Ł, Jaśkiewicz A, Nowak AS (2015) Existence of stationary Markov perfect equilibria in stochastic altruistic growth economies. J Optim Theory Appl 165(1):295–315

    Article  MathSciNet  MATH  Google Scholar 

  10. Balbus Ł, Nowak AS (2008) Existence of perfect equilibria in a class of multigenerational stochastic games of capital accumulation. Automatica 44(6):1471–1479

    Article  MathSciNet  MATH  Google Scholar 

  11. Balbus Ł, Reffett K, Woźny Ł (2012) Stationary Markovian equilibrium in altruistic stochastic OLG models with limited commitment. J Math Econ 48(2):115–132

    Article  MathSciNet  MATH  Google Scholar 

  12. Balbus Ł, Reffett K, Woźny Ł (2013a) A constructive geometrical approach to the uniqueness of Markov perfect equilibrium in stochastic games of intergenerational altruism. J Econ Dyn Control 37(5):1019–1039

    Article  Google Scholar 

  13. Balbus Ł, Reffett K, Woźny Ł (2013b) Markov stationary equilibria in stochastic supermodular games with imperfect private and public information. Dyn Games Appl 3(2):187–206

    Article  MathSciNet  MATH  Google Scholar 

  14. Balbus Ł, Reffett K, Woźny Ł (2014) A constructive study of Markov equilibria in stochastic games with strategic complementarities. J Econ Theory 150:815–840

    Article  MathSciNet  MATH  Google Scholar 

  15. Balbus Ł, Reffett K, Woźny Ł (2015) Time consistent Markov policies in dynamic economies with quasi-hyperbolic consumers. Int J Game Theory 44(1):83–112

    Article  MathSciNet  MATH  Google Scholar 

  16. Barelli P, Duggan J (2014) A note on semi-Markov perfect equilibria in discounted stochastic games. J Econ Theory 151:596–604

    Article  MathSciNet  MATH  Google Scholar 

  17. Beer G (1980) The approximation of upper semicontinuous multifunction by step multifunction. Pac J Math 87(1):11–19

    Article  MathSciNet  MATH  Google Scholar 

  18. Bernheim BD, Ray D (1983) Altruistic growth economies. I. Existence of bequest equilibria. Mimeo

  19. Bernheim BD, Ray D, Yeltekin S (2015) Poverty and self-control. Econometrica 83(5):1877–1911

  20. Billingsley P (1968) Convergence of probability measures. Wiley, New York

    MATH  Google Scholar 

  21. Caplin A, Leahy J (2006) The recursive approach to time inconsistency. J Econ Theory 131(1):134–156

    Article  MathSciNet  MATH  Google Scholar 

  22. Chade H, Prokopovych P, Smith L (2008) Repeated games with present-biased preferences. J Econ Theory 139(1):157–175

    Article  MathSciNet  MATH  Google Scholar 

  23. Chang R (1998) Credible monetary policy in an infinite horizon model: recursive approaches. J Econ Theory 81(2):431–461

    Article  MathSciNet  MATH  Google Scholar 

  24. Cole HL, Kocherlakota N (2001) Dynamic games with hidden actions and hidden states. J Econ Theory 98(1):114–126

    Article  MathSciNet  MATH  Google Scholar 

  25. Cronshaw MB (1997) Algorithms for finding repeated game equilibria. Comput Econ 10(2):139–168

    Article  MATH  Google Scholar 

  26. Curtat L (1996) Markov equilibria of stochastic games with complementarities. Games Econ Behav 17(2):177–199

    Article  MathSciNet  MATH  Google Scholar 

  27. Doraszelski U, Escobar JF (2012) Restricted feedback in long term relationships. J Econ Theory 147(1):142–161

    Article  MathSciNet  MATH  Google Scholar 

  28. Duggan J (2012) Noisy stochastic games. Econometrica 80(5):2017–2045

    Article  MathSciNet  MATH  Google Scholar 

  29. Dutta PK, Sundaram R (1992) Markovian equilibrium in a class of stochastic games: existence theorems for discounted and undiscounted models. Econ Theory 2(2):197–214

    Article  MathSciNet  MATH  Google Scholar 

  30. Feng Z, Miao J, Peralta-Alva A, Santos MS (2014) Numerical simulation of nonoptimal dynamic equilibrium models. Int Econ Rev 55(1):83–110

    Article  MathSciNet  MATH  Google Scholar 

  31. Harris C, Laibson D (2001) Dynamic choices of hyperbolic consumers. Econometrica 69(4):935–57

    Article  MathSciNet  MATH  Google Scholar 

  32. Jaśkiewicz A, Nowak AS (2014) Stationary Markov perfect equilibria in risk sensitive stochastic overlapping generations models. J Econ Theory 151:411–447

    Article  MathSciNet  MATH  Google Scholar 

  33. Jaśkiewicz A, Nowak A S (2015) Stationary almost Markov perfect equilibria in discounted stochastic games. Math Oper Res. doi:10.1287/moor.2015.0734

  34. Judd KL, Yeltekin S, Conklin J (2003) Computing supergame equilibria. Econometrica 71(4):1239–1254

    Article  MathSciNet  MATH  Google Scholar 

  35. Karp L (2005) Global warming and hyperbolic discounting. J Polit Econ 89(2–3):261–282

    Google Scholar 

  36. Krusell P, Kuruşçu B, Smith AAJ (2002) Equilibrium welfare and government policy with quasi-geometric discounting. J Econ Theory 105(1):42–72

    Article  MathSciNet  MATH  Google Scholar 

  37. Kydland F, Prescott E (1980) Dynamic optimal taxation, rational expectations and optimal control. J Econ DynControl 2(1):79–91

    Google Scholar 

  38. Laibson D (1997) Golden eggs and hyperbolic discounting. Q J Econ 112(2):443–477

    Article  MATH  Google Scholar 

  39. Levy Y, McLennan A (2015) Corrigendum to: discounted stochastic games with no stationary Nash equilibrium: two examples. Econometrica 83(3):1237–1252

    Article  MathSciNet  Google Scholar 

  40. Mertens J-F, Parthasarathy T (1987) Equilibria for discounted stochastic games. C.O.R.E. Discussion Paper 8750

  41. Mertens J-F, Parthasarathy T (2003) Equilibria for discounted stochastic games. In: Neyman A, Sorin S (eds) Stochastic games and applications. NATO Advanced Science Institutes series D: behavioural and social sciences. Kluwer, Boston

    Google Scholar 

  42. Mertens J-F, Sorin S, Zamir S (2015) Repeated games. Cambridge University Press, New York

    Book  MATH  Google Scholar 

  43. Milgrom P, Roberts J (1994) Comparing equilibria. Am Econ Rev 84(3):441–459

    Google Scholar 

  44. Nowak AS (2003) On a new class of nonzero-sum discounted stochastic games having stationary Nash equilibrium points. Int JGame Theory 32:121–132

    Article  MathSciNet  MATH  Google Scholar 

  45. Nowak AS (2007) On stochastic games in economics. Math Methods Oper Res 66(3):513–530

    Article  MathSciNet  MATH  Google Scholar 

  46. Nowak AS (2010) On a noncooperative stochastic game played by internally cooperating generations. J Optim Theory Appl 144(1):88–106

    Article  MathSciNet  MATH  Google Scholar 

  47. Nowak AS, Raghavan T (1992) Existence of stationary correlated equilibria with symmetric information for discounted stochastic games. Math Oper Res 17(3):519–526

    Article  MathSciNet  MATH  Google Scholar 

  48. Pearce D, Stacchetti E (1997) Time consistent taxation by a government with redistributive goals. J Econ Theory 72(2):282–305

    Article  MathSciNet  MATH  Google Scholar 

  49. Phelan C, Stacchetti E (2001) Sequential equilibria in a Ramsey tax model. Econometrica 69(6):1491–1518

    Article  MathSciNet  MATH  Google Scholar 

  50. Shapley L (1953) Stochastic games. Proc Natl Acad Sci USA 39(10):1095–1100

    Article  MathSciNet  MATH  Google Scholar 

  51. Sleet C, Yeltekin S (2015) On the computation of value correspondences. Dyn Games Appl doi:10.1007/s13235-015-0139-1

  52. Stokey NL (1991) Credible public policy. J Econ DynControl 15(4):627–656

    MATH  Google Scholar 

  53. Strotz RH (1955) Myopia and inconsistency in dynamic utility maximization. Rev Econ Stud 23(3):165–180

    Article  Google Scholar 

  54. Topkis DM (1978) Minimizing a submodular function on a lattice. Oper Res 26(2):305–321

    Article  MathSciNet  MATH  Google Scholar 

  55. Topkis DM (1979) Equilibrium points in nonzero-sum n-person submodular games. SIAM J Control Optim 17(6):773–787

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Mathematics, Computer Sciences and Econometrics, University of Zielona Góra, ul. Szafrana 4a, 65-516, Zielona Gora, Poland

    Łukasz Balbus

  2. Department of Quantitative Economics, Warsaw School of Economics, al. Niepodległości 162, 02-554, Warsaw, Poland

    Łukasz Woźny

Authors
  1. Łukasz Balbus
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Łukasz Woźny
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Łukasz Woźny.

Additional information

We thank Rabah Amir, Ed Prescott and Kevin Reffett as well as participants of 10th Society for the Advancement in Economic Theory Conference in Singapore (2010), NSF/NBER/CEME Conference in Mathematical Economics and General Equilibrium Theory in Iowa (2011), Paris Game Theory Seminar (2012) participants at Institut Henri Poincare, Nuffield Economic Theory Seminar (2012) at the University of Oxford and Economic Theory Seminar at the Paris School of Economics (2014) for helpful discussions during the writing of this paper. Woźny thanks the Deans Grant at WSE for financial support. All the usual caveats apply. The project was financed by NCN grant DEC-2013/11/D/HS4/03813.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Balbus, Ł., Woźny, Ł. A Strategic Dynamic Programming Method for Studying Short-Memory Equilibria of Stochastic Games with Uncountable Number of States. Dyn Games Appl 6, 187–208 (2016). https://doi.org/10.1007/s13235-015-0171-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13235-015-0171-1

Keywords

JEL Classification

Search

Navigation