Skip to main content
SpringerLink
Log in

Completely mixed strategies for single controller unichain semi-Markov games with undiscounted payoffs

  • Original Paper
  • Published:
Operational Research Aims and scope Submit manuscript

Abstract

Zero-sum two-person finite undiscounted (limiting ratio average) semi-Markov games are considered where the transition probabilities and the transition times are controlled by a fixed player in all states. We prove that if such a game is unichain, the value and optimal stationary strategies can be obtained from an optimal solution of a linear programming algorithm for the associated undiscounted unichain single controller stochastic game (obtained by a data transformation method). The single controller undiscounted unichain semi-Markov games have been formulated as a linear complementarity problem and solved using a stepwise principal pivoting algorithm. We provide necessary and sufficient conditions for such games to be completely mixed (i.e., every optimal stationary strategy for each player assigns a positive probability to every action in every state). Some properties analogous to completely mixed matrix games are also established in this paper.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Blackwell D, Girshick MA (1954) Theory of games and statistical decisions. Wiley, New York

    Google Scholar 

  • Cottle RW, Pang JS, Stone RE (1992) The linear complementarity problem. Academic, New York

    Google Scholar 

  • Federgruen A, Hordijk A, Tijms HC (1978) A note on simultaneous recurrence conditions on a set of denumerable stochastic matrices. J Appl Prob 15(4):842–847

    Article  Google Scholar 

  • Filar JA (1985) The completely mixed single-controller stochastic game. Proc Am Math Soc 95(4):585–594

    Article  Google Scholar 

  • Filar JA, Raghavan TES (1984) A matrix game solution of the single-controller stochastic game. Math Oper Res 9:356–362

    Article  Google Scholar 

  • Howard RA (1963) Semi-Markovian decision processes. Proceedings International Statistics Institute, Ottawa

    Google Scholar 

  • Jaskiewicz A (2002) Zero-sum semi-Markov games. SIAM J Control Optim 41(3):723–739

    Article  Google Scholar 

  • Jianyong L, Xiaobo Z (2004) On average reward semi-Markov decision processes with a general multichain structure. Math Oper Res 29(2):339–352

    Article  Google Scholar 

  • Jurdzinski M, Savani R (2008) A simple P-matrix linear complementarity problem for discounted games. In: Beckmann A, Dimitracopoulos C, Lwe B (eds) Lecture Notes In Computer Science, CiE 2008, LNCS 5028. Springer, Berlin, Heidelberg, pp 283–293

  • Jurg AP, Jansen MJM, Parthasarathy T, Tijs SH (1990) On weakly completely mixed bimatrix games. Linear Algebra Appl 141:61–74

    Article  Google Scholar 

  • Kaplansky I (1945) A contribution to von Neumann’s theory of games. Ann Math 46(2):474–479

    Article  Google Scholar 

  • Lal AK, Sinha S (1992) Zero-sum two-person semi-Markov games. J Appl Prob 29(1):56–72

    Article  Google Scholar 

  • Lemke CE (1965) Bimatrix equilibrium points and mathematical programming. Manag Sci 11:681–689

    Article  Google Scholar 

  • Mertens JF, Neyman A (1981) Stochastic games. Int J Game Theory 10(2):53–66

    Article  Google Scholar 

  • Mohan SR, Neogy SK, Parthasarathy T (2001) Pivoting algorithms for some classes of stochastic games: a survey. Int Game Theory Rev 3(2 & 3):253–281

    Article  Google Scholar 

  • Mondal P (2015) Linear programming and zero-sum two-person undiscounted semi-Markov games. Asia Pac J Oper Res 32(6):1550043-1–1550043-20

    Article  Google Scholar 

  • Mondal P (2016) On undiscounted semi-Markov decision processes with absorbing states. Math Methods Oper Res 83(2):161–177

    Article  Google Scholar 

  • Mondal P, Neogy SK, Sinha S, Ghorui D (2016) Completely mixed strategies for two structured classes of semi-Markov games, principal pivot transform and its generalizations. Appl Math Optim. doi:10.1007/s00245-016-9362-4

  • Mondal P, Sinha S (2015) Ordered field property for semi-Markov games when one player controls transition probabilities and transition times. Int Game Theory Rev 17(2):1540022-1–1540022-26

    Article  Google Scholar 

  • Neogy SK, Dubey D, Ghorui D (2015) Completely mixed generalized bimatrix game, vertical linear complementarity problem and completely mixed switching controller stochastic game. Preprints, isid/SQCORD/2015/03

  • Neogy SK, Das AK, Gupta A (2012) Generalized principal pivot transforms, complementarity theory and their applications in stochastic games. Optim Lett 6:339–356

    Article  Google Scholar 

  • Parsons TD (1970) Applications of principal pivoting. In: Kuhn HW (ed) Proceedings of the Princeton symposium on mathematical programming. Princeton University Press, Princeton, pp 561–581

    Google Scholar 

  • Parthasarathy T, Raghavan TES (1981) An orderfield property for stochastic games when one player controls transition probabilities. J Optim Theory Appl 33(3):375–392

    Article  Google Scholar 

  • Raghavan TES (1970) Completely mixed strategies in bimatrix games. J Lond Math Soc 2(2):709–712

    Article  Google Scholar 

  • Raghavan TES, Filar JA (1991) Algorithms for stochastic games—a survey. Math Methods Oper Res 35(6):437–472

    Article  Google Scholar 

  • Ross SM (1970) Applied probability models with optimization applications. Holden-Day, San Francisco

    Google Scholar 

  • Schultz TA (1992) Linear complementarity and discounted switching controller stochastic games. J Optim Theory Appl 73(1):89–99

    Article  Google Scholar 

  • Schweitzer PJ (1971) Iterative solution of the functional equations of undiscounted Markov renewal programming. J Math Anal App 34(3):495–501

    Article  Google Scholar 

  • Shapley L (1953) Stochastic games. Proc Natl Acad Sci 39(10):1095–1100

    Article  Google Scholar 

  • Shapley LS, Snow RN (1950) Basic solutions of discrete games. Annals of mathematics studies, vol 24. Princeton University Press, Princeton

    Google Scholar 

  • Tsatsomeros M (2000) Principal pivot transforms: properties and applications. Linear Algebra Appl 307:151–165

    Article  Google Scholar 

  • Tucker AW (1963) Principal pivotal transforms of square matrices. SIAM Rev 5:305

    Google Scholar 

  • Vega-Amaya O (2003) Zero-sum average semi-Markov games: fixed-point solutions of the Shapley equation. SIAM J Control Optim 42(5):1876–1894

    Article  Google Scholar 

  • Vrieze OJ (1981) Linear programming and undiscounted stochastic games in which one player controls transitions. OR Spektrum 3(1):29–35

    Article  Google Scholar 

Download references

Acknowledgments

The author wishes to thank the unknown referees who have patiently gone through this paper and whose constructive suggestions have improved its presentation and readability considerably.

Author information

Authors and Affiliations

  1. Mathematics Department, Chandernagore College, Government of West Bengal, Hooghly, 712136, India

    Prasenjit Mondal

Authors
  1. Prasenjit Mondal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Prasenjit Mondal.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mondal, P. Completely mixed strategies for single controller unichain semi-Markov games with undiscounted payoffs. Oper Res Int J 18, 451–468 (2018). https://doi.org/10.1007/s12351-016-0272-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12351-016-0272-7

Keywords

Mathematics Subject Classification

Search

Navigation