Skip to main content
SpringerLink
Log in

Optimality in transient markov chains and linear programming

  • Published:
Mathematical Programming Submit manuscript

Abstract

It is shown how a discrete Markov programming problem can be transformed, using a linear program, into an equivalent problem from which the optimal decision rule can be trivially deduced. This transformation is applied to problems which have either transient probabilities or discounted costs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M.L. Balinski, “Notes on a constructive approach to linear programming”, in: G.B. Dantzig and A.F. Veinott, eds.,Mathematics of the decision sciences (Am. Math. Soc., Providence, RI, 1968) pp. 38–64.

    Google Scholar 

  2. D. Blackwell, “Discrete dynamic programming”,Annals of Mathematical Statistics 33 (1962) 719–726.

    Google Scholar 

  3. F. D'Épenoux, “Sur un problème de production et de stockage dans l'aléatoire”,Revue Francaise de Recherche Opérationelle 14 (1960) 3–16.

    Google Scholar 

  4. G.T. DeGhellinck and C.D. Eppen, “Linear programming solutions for separable Markov decision problems”,Management Science 13 (1967) 371–394.

    Google Scholar 

  5. E.V. Denardo, “Contraction mappings in the theory underlying dynamic programming”,SIAM Review 9 (1967) 165–177.

    Google Scholar 

  6. R.A. Howard,Dynamic programming and Markov processes (Wiley, New York, 1960).

    Google Scholar 

  7. H.W. Kuhn, “The Hungarian method for the assignment problem”,Naval Research Logistics Quarterly 2 (1955) 83–97.

    Google Scholar 

  8. A.S. Manne, “Linear programming and sequential decisions”,Management Science 6 (1960) 259–267.

    Google Scholar 

  9. A.F. Veinott, Jr., “Markov decision chains”, in: G.B. Dantzig and B.C. Eaves, eds.,Studies in optimization (Mathematical Association of America, Washington, DC, 1974) pp. 124–159.

    Google Scholar 

  10. P. Wolfe and G.B. Dantzig, “Linear programming in a Markov chain”,Operations Research 10 (1962) 702–710.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mathematics Department, Dalhousie University Halifax, B3H 4H8, Nova Scotia, Canada

    W. R. S. Sutherland

Authors
  1. W. R. S. Sutherland
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

This research was supported by the National Research Council of Canada, Grant A7751.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sutherland, W.R.S. Optimality in transient markov chains and linear programming. Mathematical Programming 18, 1–6 (1980). https://doi.org/10.1007/BF01588291

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01588291

Key words

Search

Navigation