Skip to main content
SpringerLink
Log in

Stochastic Games with Unbounded Payoffs: Applications to Robust Control in Economics

  • Published:
Dynamic Games and Applications Aims and scope Submit manuscript

Abstract

We study a discounted maxmin control problem with general state space. The controller is unsure about his model in the sense that he also considers a class of approximate models as possibly true. The objective is to choose a maxmin strategy that will work under a range of different model specifications. This is done by dynamic programming techniques. Under relatively weak conditions, we show that there is a solution to the optimality equation for the maxmin control problem as well as an optimal strategy for the controller. These results are applied to the theory of optimal growth and the Hansen–Sargent robust control model in macroeconomics. We also study a class of zero-sum discounted stochastic games with unbounded payoffs and simultaneous moves and give a brief overview of recent results on stochastic games with weakly continuous transitions and the limiting average payoffs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alvarez F, Stokey N (1998) Dynamic programming with homogeneous functions. J Econ Theory 82:167–189

    Article  MathSciNet  MATH  Google Scholar 

  2. Başar T, Bernhard P (1995) H ∞-optimal control and related minimax design problems. Birkhäuser, Boston

    MATH  Google Scholar 

  3. Berge C (1963) Topological spaces. MacMillan, New York

    MATH  Google Scholar 

  4. Bertsekas DP, Shreve SE (1978) Stochastic optimal control: the discrete-time case. Academic Press, New York

    MATH  Google Scholar 

  5. Bhattacharya R, Majumdar M (2007) Random dynamical systems: theory and applications. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  6. Blackwell D (1965) Discounted dynamic programming. Ann Math Stat 36:226–235

    Article  MathSciNet  MATH  Google Scholar 

  7. Brown LD, Purves R (1973) Measurable selections of extrema. Ann Stat 1:902–912

    Article  MathSciNet  MATH  Google Scholar 

  8. Couwenbergh HAM (1980) Stochastic games with metric state spaces. Int J Game Theory 9:25–36

    Article  MathSciNet  MATH  Google Scholar 

  9. Durán J (2000) On dynamic programming with unbounded returns. Econom Theory 15:339–352

    Article  MathSciNet  MATH  Google Scholar 

  10. Durán J (2003) Discounting long-run average growth in stochastic dynamic programs. Econom Theory 22:395–413

    Article  MathSciNet  MATH  Google Scholar 

  11. Fan K (1953) Minimax theorems. Proc Natl Acad Sci USA 39:42–47

    Article  MATH  Google Scholar 

  12. González-Trejo JI, Hernández-Lerma O, Hoyos-Reyes LF (2003) Minimax control of discrete-time stochastic systems. SIAM J Control Optim 41:1626–1659

    Article  MATH  Google Scholar 

  13. Hansen LP, Sargent TJ (2008) Robustness. Princeton University Press, Princeton

    MATH  Google Scholar 

  14. Hansen LP, Sargent TJ (2010) Wanting robustness in macroeconomics. In: Friedman BM, Woodford M (eds) Handbook of monetary economics, vol 3, pp 1097–1157

    Chapter  Google Scholar 

  15. Hansen LP, Sargent TJ, Tallarini TD (1999) Robust permanent income and pricing. Rev Econ Stud 66:873–907

    Article  MATH  Google Scholar 

  16. Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes: basic optimality criteria. Springer, New York

    Google Scholar 

  17. Hernández-Lerma O, Lasserre JB (1999) Further topics on discrete-time Markov control processes. Springer, New York

    MATH  Google Scholar 

  18. Himmelberg CJ (1975) Measurable relations. Fundam Math 87:53–72

    MathSciNet  MATH  Google Scholar 

  19. Himmelberg CJ, Van Vleck FS (1975) Multifunctions with values in a space of probability measures. J Math Anal Appl 50:108–112

    Article  MathSciNet  MATH  Google Scholar 

  20. Iyengar GN (2005) Robust dynamic programming. Math Oper Res 30:257–280

    Article  MathSciNet  MATH  Google Scholar 

  21. Jaśkiewicz A (2009) Zero-sum ergodic semi-Markov games with weakly continuous transition probabilities. J Optim Theory Appl 141:321–348

    Article  MathSciNet  MATH  Google Scholar 

  22. Jaśkiewicz A (2010) On a continuous solution to the Bellman–Poisson equation in stochastic games. J Optim Theory Appl 145:451–458

    Article  MathSciNet  MATH  Google Scholar 

  23. Jaśkiewicz A, Nowak AS (2006) Zero-sum ergodic stochastic games with Feller transition probabilities. SIAM J Control Optim 45:773–789

    Article  MathSciNet  MATH  Google Scholar 

  24. Jaśkiewicz A, Nowak AS (2011) Discounted dynamic programming with unbounded returns: application to economic models. J Math Anal Appl 378:450–462

    Article  MATH  Google Scholar 

  25. Kamihigashi T (2007) Stochastic optimal growth with bounded and unbounded utility and with bounded or unbounded shocks. J Math Econ 43:477–500

    Article  MathSciNet  MATH  Google Scholar 

  26. Karatzas I, Sudderth WD (2010) Two characterizations of optimality in dynamic programming. Appl Math Optim 61:421–434

    Article  MathSciNet  MATH  Google Scholar 

  27. Kuratowski K, Ryll-Nardzewski C (1965) A general theorem on selectors. Bull Acad Pol Sci (Ser Math) 13:397–403

    MathSciNet  MATH  Google Scholar 

  28. Küenle HU (1986) Stochastische Spiele und Entscheidungsmodelle. BG Teubner, Leipzig

    MATH  Google Scholar 

  29. Küenle HU (2007) On Markov games with average reward criterion and weakly continuous transition probabilities. SIAM J Control Optim 45:2156–2168

    Article  MathSciNet  MATH  Google Scholar 

  30. Kurano M (1987) Minimax strategies for average cost stochastic games with an application to inventory models. J Oper Res Soc Jpn 30:232–247

    MathSciNet  MATH  Google Scholar 

  31. Le Van C, Morhaim L (2002) Optimal growth models with bounded or unbounded returns: a unifying approach. J Econ Theory 105:158–187

    Article  MATH  Google Scholar 

  32. Ljungqvist L, Sargent TJ (2000) Recursive macroeconomic theory. MIT Press, Cambridge

    Google Scholar 

  33. Maitra A, Parthasarathy T (1970) On stochastic games. J Optim Theory Appl 5:289–300

    Article  MathSciNet  MATH  Google Scholar 

  34. Maitra A, Sudderth W (1993) Borel stochastic games with limsup payoffs. Ann Probab 21:861–885

    Article  MathSciNet  MATH  Google Scholar 

  35. Maccheroni F, Marinacci M, Rustichini A (2006) Dynamic variational preferences. J Econ Theory 128:4–44

    Article  MathSciNet  MATH  Google Scholar 

  36. Martins-da-Rocha VF, Vailakis Y (2010) Existence and uniqueness of fixed-point for local contractions. Econometrica 78:1127–1141

    Article  MathSciNet  MATH  Google Scholar 

  37. Matkowski J, Nowak AS (2011) On discounted dynamic programming with unbounded returns. Econom Theory 46:455–474

    Article  MATH  Google Scholar 

  38. Mertens JF, Neyman A (1981) Stochastic games. Int J Game Theory 10:53–66

    Article  MathSciNet  MATH  Google Scholar 

  39. Meyn SP, Tweedie RL (1993) Markov chains and stochastic stability. Springer, New York

    MATH  Google Scholar 

  40. Meyn SP, Tweedie RL (1994) Computable bounds for geometric convergence rates of Markov chains. Ann Appl Probab 4:981–1011

    Article  MathSciNet  MATH  Google Scholar 

  41. Neveu J (1965) Mathematical foundations of the calculus of probability. Holden-Day, San Francisco

    MATH  Google Scholar 

  42. Nowak AS (1985) Universally measurable strategies in zero-sum stochastic games. Ann Probab 13:269–287

    Article  MathSciNet  MATH  Google Scholar 

  43. Nowak AS (1986) Semicontinuous nonstationary stochastic games. J Math Anal Appl 117:84–99

    Article  MathSciNet  MATH  Google Scholar 

  44. Nowak AS (2010) On measurable minimax selectors. J Math Anal Appl 366:385–388

    Article  MathSciNet  MATH  Google Scholar 

  45. Parthasarathy KR (1967) Probability measures on metric spaces. Academic Press, New York

    MATH  Google Scholar 

  46. Petersen IR, James MR, Dupuis P (2000) Minimax optimal control of stochastic uncertain systems with relative entropy constraints. IEEE Trans Autom Control 45:398–412

    Article  MathSciNet  MATH  Google Scholar 

  47. Puterman M (1980) Markov decision processes: discrete stochastic dynamic programming. Wiley-Interscience, New York

    Google Scholar 

  48. Rincón-Zapatero JP, Rodriguez-Palmero C (2003) Existence and uniqueness of solutions to the Bellman equation in the unbounded case. Econometrica 71:1519–1555. Corrigendum Econometrica 77:317–318

    Article  MathSciNet  MATH  Google Scholar 

  49. Schäl M (1975) Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal. Z Wahrscheinlichkeitstheor Verw Geb 32:179–196

    Article  MATH  Google Scholar 

  50. Shapley LS (1953) Stochastic games. Proc Natl Acad Sci USA 39:1095–1100

    Article  MathSciNet  MATH  Google Scholar 

  51. Stokey NL, Lucas RE, Prescott E (1989) Recursive methods in economic dynamics. Harvard University Press, Cambridge

    MATH  Google Scholar 

  52. Strauch R (1966) Negative dynamic programming. Ann Math Stat 37:871–890

    Article  MathSciNet  MATH  Google Scholar 

  53. Wessels J (1977a) Markov programming by successive approximations with respect to weighted supremum norms. J Math Anal Appl 58:326–335

    Article  MathSciNet  MATH  Google Scholar 

  54. Wessels J (1977b) Markov games with unbounded rewards. In: Schäl M (ed) Dynamische Optimierung, Bonn, 1977. Bonner Mathematische Schriften, vol 98, pp 133–147

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Mathematics and Computer Science, Wrocław University of Technology, Wybrzeże Wyspiańskiego 27, 50-370, Wrocław, Poland

    Anna Jaśkiewicz

  2. Faculty of Mathematics, Computer Science and Econometrics, University of Zielona Góra, Podgórna 50, 65-246, Zielona Góra, Poland

    Andrzej S. Nowak

  3. Institute of Finance, PWSZ Nysa, Poland

    Andrzej S. Nowak

Authors
  1. Anna Jaśkiewicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrzej S. Nowak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrzej S. Nowak.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jaśkiewicz, A., Nowak, A.S. Stochastic Games with Unbounded Payoffs: Applications to Robust Control in Economics. Dyn Games Appl 1, 253–279 (2011). https://doi.org/10.1007/s13235-011-0013-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13235-011-0013-8

Keywords

Search

Navigation