Skip to main content
SpringerLink
Log in

Nonlinear programming and stationary equilibria in stochastic games

  • Published:
Mathematical Programming Submit manuscript

Abstract

Stationary equilibria in discounted and limiting average finite state/action space stochastic games are shown to be equivalent to global optima of certain nonlinear programs. For zero sum limiting average games, this formulation reduces to a linear objective, nonlinear constraints program, which finds the “best” stationary strategies, even whenε-optimal stationary strategies do not exist, for arbitrarily smallε.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. T. Bewley and E. Kohlberg, “On stochastic games with stationary optimal strategies,”Mathematics of Operations Research 1 (1978) 104–125.

    Google Scholar 

  2. M. Breton, A. Haurie, J.A. Filar and T. Schultz, “On the computation of equilibria in discounted stochastic dynamic games,” in: T. Basar, M. Beckmann and W. Krelle, eds.,Lecture notes in Economics and Mathematical Systems No. 265 (Springer, Berlin, 1986) pp. 64–87.

    Google Scholar 

  3. D. Blackwell, “Discrete dynamic programming,”Annals of Mathematical Statistics 33 (1962) 719–726.

    Google Scholar 

  4. D. Blackwell and T.S. Ferguson, “The big match,”Annals of Mathematical Statistics 39 (1968) 159–163.

    Google Scholar 

  5. C. Derman,Finite State Markovian Decision Processes (Academic Press, New York, 1970).

    Google Scholar 

  6. J.A. Filar and T. Schultz, “Nonlinear programming and stationary strategies in stochastic games,”Mathematical Programming 35 (1986) 243–247.

    Google Scholar 

  7. A.M. Fink, “Equilibrium in a stochastic n-person game,”Journal of Science of Hiroshima University, Series A-I 28 (1964) 89–93.

    Google Scholar 

  8. D. Gillete, “Stochastic games with zero stop probabilities,” in: M. Dresher, A.W. Tucker and P. Wolfe, eds.,Contributions to the Theory of Games III (Princeton University Press, Princeton, NJ, 1957) pp. 179–187.

    Google Scholar 

  9. A. Hordijk and L.C.M. Kallenberg, “Linear programming and Markov decision chains,”Management Science 25 (1979) 352–362.

    Google Scholar 

  10. A. Hordijk, O.J. Vrieze and G.L. Wanrooy, “Semi-Markov strategies in stochastic games,”International Journal of Game Theory 12 (1983) 81–89.

    Google Scholar 

  11. C.E. Lemke and J. Howson, “Equilibrium points of bimatrix games,”Journal of the Society of Industrial and Applied Mathematics 12 (1964) 413–423.

    Google Scholar 

  12. O.L. Mangasarian and H. Stone, “Two-person nonzero-sum games and quadratic programming,”Journal of Mathematical Analysis and Applications 9 (1964) 348–355.

    Google Scholar 

  13. P.D. Rogers, “Non-zero-sum stochastic games,” PhD Thesis, Report ORC 69-8 Operations Research Center, University of California (Berkeley, CA, 1969).

    Google Scholar 

  14. J. Rosenmuller, “On a generalization of the Lemke-Howson algorithm to noncooperativen-person games,”SIAM Journal of Applied Mathematics 21 (1971) 73–79.

    Google Scholar 

  15. U.G. Rothblum, “Solving stopping stochastic games by maximizing a linear function subject to quadratic constraints,” in: O. Moeschlin and D. Pallashke, eds.,Game Theory and Related Topics (North-Holland, Amsterdam, 1979) pp. 103–105.

    Google Scholar 

  16. T. Schultz, “Mathematical programming and stochastic games,” PhD Thesis, Johns Hopkins University (Baltimore, MD, 1987).

    Google Scholar 

  17. M. Sobel, “Noncooperative stochastic games,”The Annals of Mathematical Statistics 42 (1971) 1930–1935.

    Google Scholar 

  18. M. Takahashi, “Equilibrium points of stochastic, noncooperative n-person games,”Journal of Science of Hiroshima University Series A-I 28 (1964) 95–99.

    Google Scholar 

  19. O.J. Vrieze,Stochastic Games with Finite State and Action Spaces, CWI-tract No. 33, (Centre for Mathematics and Computer Science, Amsterdam, 1987).

    Google Scholar 

  20. R. Wilson, “Computing Equilibria ofn-person games,”SIAM Journal of Applied Mathematics 21 (1971) 80–87.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Maryland, 21228, Baltimore County, MD, USA

    J. A. Filar

  2. School of Business Administration, Augusta College, 61201, Rock Island, IL, USA

    T. A. Schultz

  3. Department of Mathematics, University of Limburg, P.O. Box 616, 6200 MD, Maastricht, Netherlands

    F. Thuijsman & O. J. Vrieze

Authors
  1. J. A. Filar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. T. A. Schultz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. F. Thuijsman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. O. J. Vrieze
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

The work of the first author was supported in part by the Air Force Office of Scientific Research, and by the National Science Foundation under Grant No ECS-8704954.

The work of the third author was supported by The Netherlands Organization for Scientific Research NWO, project 10-64-10.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Filar, J.A., Schultz, T.A., Thuijsman, F. et al. Nonlinear programming and stationary equilibria in stochastic games. Mathematical Programming 50, 227–237 (1991). https://doi.org/10.1007/BF01594936

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01594936

Key words

Search

Navigation