Skip to main content
SpringerLink
Log in

Zero-Sum Markov Games with Random State-Actions-Dependent Discount Factors: Existence of Optimal Strategies

  • Published:
Dynamic Games and Applications Aims and scope Submit manuscript

Abstract

This work deals with a class of discrete-time zero-sum Markov games under a discounted optimality criterion with random state-actions-dependent discount factors of the form \(\tilde{\alpha }(x_{n},a_{n},b_{n},\xi _{n+1})\), where \(x_{n}, a_{n}, b_{n}\), and \(\xi _{n+1}\) are the state, the actions of players, and a random disturbance at time n, respectively, taking values in Borel spaces. Assuming possibly unbounded payoff, we prove the existence of a value of the game as well as a stationary pair of optimal strategies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ash RB, Doléans-Dade C (2000) Probability and measure theory, 2nd edn. Academic Press, London

    MATH  Google Scholar 

  2. Bäuerle N, Rieder U (2011) Markov decision processes with applications to finance. Springer, Berlin

    Book  MATH  Google Scholar 

  3. Carmon Y, Shwartz A (2009) Markov decision processes with exponentially representable discounting. Oper Res Lett 37(1):51–55

    Article  MathSciNet  MATH  Google Scholar 

  4. Cruz-Suárez H, Ilhuicatzi-Roldán R, Montes-de Oca R (2014) Markov decision processes on Borel spaces with total cost and random horizon. J Optim Theory Appl 162(1):329–346

    Article  MathSciNet  MATH  Google Scholar 

  5. Dutta PK, Sundaram R (1992) Markovian equilibrium in a class of stochastic games: existence theorems for discounted and undiscounted models. Econ Theory 2(2):197–214

    Article  MathSciNet  MATH  Google Scholar 

  6. Dynkin EB, Yushkevich AA (1979) Controlled Markov processes. Springer, Berlin

    Book  Google Scholar 

  7. Engwerda J (2005) LQ dynamic optimization and differential games. Wiley, New York

    Google Scholar 

  8. Feinberg EA, Shwartz A (1999) Constrained dynamic programming with two discount factors: applications and an algorithm. IEEE Trans Autom Control 44(3):628–631

    Article  MathSciNet  MATH  Google Scholar 

  9. Filar J, Vrieze K (2012) Competitive Markov decision processes. Springer, Berlin

    MATH  Google Scholar 

  10. González-Hernández J, López-Martínez RR, Minjárez-Sosa JA (2008) Adaptive policies for stochastic systems under a randomized discounted cost criterion. Bol Soc Mat Mexicana 3(14):149–163

    MathSciNet  MATH  Google Scholar 

  11. González-Hernández J, López-Martínez RR, Minjárez-Sosa JA (2009) Approximation, estimation and control of stochastic systems under a randomized discounted cost criterion. Kybernetika 45(5):737–754

    MathSciNet  MATH  Google Scholar 

  12. González-Hernández J, López-Martínez RR, Minjárez-Sosa JA, Gabriel-Arguelles JR (2013) Constrained Markov control processes with randomized discounted cost criteria: occupation measures and extremal points. Risk Decis Anal 4(3):163–176

    Article  MATH  Google Scholar 

  13. He W, Sun Y (2017) Stationary Markov perfect equilibria in discounted stochastic games. J Econ Theory 169:35–61

    Article  MathSciNet  MATH  Google Scholar 

  14. Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes: basic optimality criteria, vol 30. Springer, Berlin

    Book  MATH  Google Scholar 

  15. Huang Y, Guo X (2012) Constrained optimality for first passage criteria in semi-Markov decision processes. In: Hernández-Hernández D, Minjárez-Sosa JA (eds) Optimization, control, and applications of stochastic systems, systems & control: foundations & applications. Birkhauser, Boston, pp 181–202 chap. 11

    Chapter  Google Scholar 

  16. Huang Y, Wei Q, Guo X (2013) Constrained Markov decision processes with first passage criteria. Ann Oper Res 206(1):197–219

    Article  MathSciNet  MATH  Google Scholar 

  17. Jaśkiewicz A, Nowak AS (2006) Approximation of noncooperative semi-Markov games. J Optim Theory Appl 131(1):115–134

    Article  MathSciNet  MATH  Google Scholar 

  18. Jaśkiewicz A, Nowak AS (2006) Zero-sum ergodic stochastic games with Feller transition probabilities. SIAM J Control Optim 45(3):773–789

    Article  MathSciNet  MATH  Google Scholar 

  19. Krausz A, Rieder U (1997) Markov games with incomplete information. Math Methods Oper Res 46(2):263–279

    Article  MathSciNet  MATH  Google Scholar 

  20. Luque-Vásquez F (2002) Zero-sum semi-Markov games in Borel spaces: discounted and average payoff. Bol Soc Mat Mexicana 8:227–241

    MathSciNet  MATH  Google Scholar 

  21. Maitra A, Parthasarathy T (1970) On stochastic games. J Optim Theory Appl 5(4):289–300

    Article  MATH  MathSciNet  Google Scholar 

  22. Maschler M, Solan E, Zamir S (2013) Game theory. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  23. Minjárez-Sosa JA (2015) Markov control models with unknown random state-action-dependent discount factors. Top 23(3):743–772

    Article  MathSciNet  MATH  Google Scholar 

  24. Minjárez-Sosa JA, Luque-Vásquez F (2008) Two person zero-sum semi-Markov games with unknown holding times distribution on one side: a discounted payoff criterion. Appl Math Optim 57(3):289–305

    Article  MathSciNet  MATH  Google Scholar 

  25. Minjárez-Sosa JA, Vega-Amaya O (2009) Asymptotically optimal strategies for adaptive zero-sum discounted Markov games. SIAM J Control Optim 48(3):1405–1421

    Article  MathSciNet  MATH  Google Scholar 

  26. Minjárez-Sosa JA, Vega-Amaya O (2013) Optimal strategies for adaptive zero-sum average Markov games. J Math Anal Appl 402(1):44–56

    Article  MathSciNet  MATH  Google Scholar 

  27. Neyman A, Sorin S (2003) Stochastic games and applications, vol 570. Kluwer, Dordrecht

    Book  MATH  Google Scholar 

  28. Nowak AS (1984) On zero-sum stochastic games with general state space. I. Prob Math Stat 4(1):13–32

    MathSciNet  MATH  Google Scholar 

  29. Nowak AS (1985) Measurable selection theorems for minimax stochastic optimization problems. SIAM J Control Optim 23(3):466–476

    Article  MathSciNet  MATH  Google Scholar 

  30. Nowak AS (1987) Nonrandomized strategy equilibria in noncooperative stochastic games with additive transition and reward structure. J Optim Theory Appl 52(3):429–441

    Article  MathSciNet  MATH  Google Scholar 

  31. Nowak AS, Szajowski K (1999) Nonzero-sum stochastic games. In: Stochastic and differential games. Annals of the international society of dynamic games, vol 4, chap 7. Springer, Berlin, pp 297–342

  32. Osborne MJ, Rubinstein A (1994) A course in game theory. MIT Press, Cambridge

    MATH  Google Scholar 

  33. Rieder U (1991) Non-cooperative dynamic games with general utility functions. In: Raghavan TES, Ferguson TS, Parthasarathy T, Vrieze OJ (eds) Stochastic games and related topics, theory and decision library, vol 7. Springer, Berlin, pp 161–174

    Chapter  Google Scholar 

  34. Schäl M (1975) Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal. Probab Theory Rel Fields 32(3):179–196

    MathSciNet  MATH  Google Scholar 

  35. Shapley LS (1953) Stochastic games. Proc Natl Acad Sci USA 39(10):1095–1100

    Article  MathSciNet  MATH  Google Scholar 

  36. Shimkin N, Shwartz A (1995) Asymptotically efficient adaptive strategies in repeated games. Part I: certainty equivalence strategies. Math Oper Res 20(3):743–767

    Article  MathSciNet  MATH  Google Scholar 

  37. Shimkin N, Shwartz A (1996) Asymptotically efficient adaptive strategies in repeated games. Part II: asymptotic optimality. Math Oper Res 21(2):487–512

    Article  MathSciNet  MATH  Google Scholar 

  38. Wei Q, Guo X (2011) Markov decision processes with state-dependent discount factors and unbounded rewards/costs. Oper Res Lett 39(5):369–374

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CONACYT–Universidad de Sonora, Rosales s/n, 83000, Hermosillo, Sonora, Mexico

    David González-Sánchez

  2. Departamento de Matemáticas, Universidad de Sonora, Rosales s/n, 83000, Hermosillo, Sonora, Mexico

    Fernando Luque-Vásquez & J. Adolfo Minjárez-Sosa

Authors
  1. David González-Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fernando Luque-Vásquez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. Adolfo Minjárez-Sosa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. Adolfo Minjárez-Sosa.

Additional information

Work supported by Consejo Nacional de Ciencia y Tecnología (CONACYT) under Grant CB2015/254306.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

González-Sánchez, D., Luque-Vásquez, F. & Minjárez-Sosa, J.A. Zero-Sum Markov Games with Random State-Actions-Dependent Discount Factors: Existence of Optimal Strategies. Dyn Games Appl 9, 103–121 (2019). https://doi.org/10.1007/s13235-018-0248-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13235-018-0248-8

Keywords

Mathematics Subject Classification

Search

Navigation