Skip to main content
SpringerLink
Log in

A simple suboptimal algorithm for system maintenanceunder partial observability

  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

We suggest a heuristic solution procedure for Partially Observable Markov Decision Processes with finite action space and finite state space with infinite horizon. The algorithm is a fast, very simple general heuristic; it is applicable for multiple states (not necessarily ordered) multiple actions and various distribution functions. The quality of the algorithm is checked in this paper against existing analytical and empirical results for two specific models of machine replacement. One model refers to the case of two‐action and two‐system states with uniform observations (Grosfeld‐Nir [4]), and the other model refers to a case of many ordered states with binomial observations (Sinuany‐Stern et al. [11]). The paper also presents the model realization for various probability distribution functions applied to maintenance and quality control.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. C.S. Albright, Structural results for partially observable Markov decision processes, Oper. Res. 27(1979)1041-1053.

    Article  Google Scholar 

  2. D.P. Bertsekas, Dynamic Programming and Stochastic Control, Academic Press, New York, 1976.

    Google Scholar 

  3. A.J. Duncan, Quality Control and Industrial Statistics, Irwin, Homewood, IL, 1986, 5th ed.

    Google Scholar 

  4. A. Grosfeld-Nir, A two-state partially observable Markov decision process with uniformly distributed observations, Oper. Res. 44(1996)458-463.

    Article  Google Scholar 

  5. S.H. Kim and J.B. Ho, A partially observable Markov decision process with lagged information, J. Oper. Res. Soc. 38(1987)439-446.

    Article  Google Scholar 

  6. D.E. Lane, A partially observable model of decision making for fishermen, Oper. Res. 37(1989)240-254.

    Article  Google Scholar 

  7. W.S. Lovejoy, A survey of algorithmic methods for POMDPs, Ann. Oper. Res. 28 (1991)47-65.

    Article  Google Scholar 

  8. W.S. Lovejoy, Computationally feasible bounds for POMDPs, Oper. Res. 39(1991)162-175.

    Article  Google Scholar 

  9. G.E. Monahan, A survey of POMDPs: Theory, models and algorithms. Manag. Sci. 28(1982)1-16.

    Article  Google Scholar 

  10. Z. Sinuany-Stern, Replacement policy under partially observed Markov process, Inter. J. Prod. Econ. 29(1993)159-166.

    Article  Google Scholar 

  11. Z. Sinuany-Stern, I. David and S. Biran, An efficient heuristic for a partially observable Markov decision process of machine replacement, Computers Oper. Res. 24(1997)117-126.

    Article  Google Scholar 

  12. C.C. White, A Markov quality control process subject to partial observation, Manag. Sci. 23(1977)843-852.

    Article  Google Scholar 

  13. C.C. White, Optimal control limit strategies for a partially observed replacement problem, Int. J. Sys. Sci. 10(1979)321-331.

    Article  Google Scholar 

  14. C.C. White, A survey of solution techniques for the POMDP, Ann. Oper. Res. 32(1991)215-230.

    Article  Google Scholar 

  15. C.C. White and D.P. Harrington, Application of Jensen's inequality for adaptive suboptimal design, J. Optim. Theory Appl. 32(1980)89-99.

    Article  Google Scholar 

  16. C.C. White and W.T. Scherer, Finite-memory suboptimal design for Partially Observed Markov Decision Processes, Oper. Res. 42(1994)439-455.

    Article  Google Scholar 

  17. D.J. White, Real applications of Markov decision processes, Interfaces 15(1985)7-83.

    Article  Google Scholar 

  18. D.J. White, Further real applications of Markov decision processes, Interfaces 18(1988)55-61.

    Article  Google Scholar 

Download references

Authors
  1. I. David
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. L. Friedman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Z. Sinuany‐Stern
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

David, I., Friedman, L. & Sinuany‐Stern, Z. A simple suboptimal algorithm for system maintenanceunder partial observability. Annals of Operations Research 91, 25–40 (1999). https://doi.org/10.1023/A:1018949723461

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1018949723461

Keywords

Search

Navigation