Skip to main content
SpringerLink
Log in

A set of successive approximation methods for discounted Markovian decision problems

  • Published:
Zeitschrift für Operations Research Aims and scope Submit manuscript

Summary

A method of successive approximations for discountedMarkovian decision problems is described byMacQueen [1966]. This paper presents a set of methods includingMacQueen's improved version of the standard dynamic programming iterative scheme. While furthermore, by the fact that we used a somewhat different approach, the physical meaning of some aspects of the successive approximation methods will probably be more transparent. Some numerical results will be given.

Zusammenfassung

Für diskontierteMarkoff-Entscheidungsprozesse hatMacQueen [1966] eine Methode der sukzessiven Annäherung beschrieben. Die vorliegende Arbeit stellt einen Satz von Methoden vor, derMacQueens verbesserte Version des iterativen Schemas der klassischen Dynamischen Programmierung enthält. Darüber hinaus wird über den hier gewählten unterschiedlichen Ansatz versucht, die praktische Bedeutung einiger Aspekte der Methode der sukzessiven Annäherung transparenter zu machen. Einige numerische Beispiele werden vorgestellt.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Blackwell, D.: Discrete dynamic programming. Ann. Math. Stat.33, 719–729, 1962.

    Google Scholar 

  • —: Discounted dynamic programming. Ann. Math. Stat.36, 226–234, 1965.

    Google Scholar 

  • Denardo, E.: Contraction mappings in the theory underlying dynamic programming. SIAM Review9, 165–177, 1967.

    Google Scholar 

  • van Doorn, E.: Successieve approximatiemethoden voor Markov beslissingsprocessen met verdiskontering. Memorandum COSOR 73-02, Eindhoven University of Technology, Department of Mathematics, Eindhoven.

  • de Ghellinck, G., andG. Eppen: Linear programming solutions for separable Markovian decision problems. Management Science13, 371–394, 1967.

    Google Scholar 

  • Grinold, R.: Elimination of suboptimal actions in Markov decision problems. Operat. Res.21, 848 to 851, 1973.

    Google Scholar 

  • Hastings, N.: The repair limit replacement method. Operat. Res.19, 337–349, 1969.

    Google Scholar 

  • Howard, R.: Dynamic programming and Markov processes. Cambridge 1960.

  • MacQueen, J.: A modified dynamic programming method for Markovian decision problems. J. Math. An. Appl.14, 38–43, 1966.

    Google Scholar 

  • —: A test for suboptimal actions in Markovian decision problems. Operat. Res.15, 559–561, 1967.

    Google Scholar 

  • Mine, H., andS. Osaki: Markovian decision processes. New York 1970.

  • Porteus, E.: Some bounds for discounted sequential decision processes. Man. Sci.18, 7–11, 1971.

    Google Scholar 

  • Wessels, J., andJ. van Nunen: Discounted semi-Markov decision processes: linear programming and policy iteration. Statistica Neerlandica29, 1–7, 1975.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Technological University Eindhoven, PO Box 513, Eindhoven, Netherlands

    J. A. E. E. van Nunen

Authors
  1. J. A. E. E. van Nunen
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

van Nunen, J.A.E.E. A set of successive approximation methods for discounted Markovian decision problems. Zeitschrift für Operations Research 20, 203–208 (1976). https://doi.org/10.1007/BF01920264

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01920264

Keywords

Search

Navigation