Skip to main content
SpringerLink
Log in

Discounted Markov decision processes with fuzzy costs

  • Original Research
  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

Fuzzy theory is a discipline that has recently appeared in the mathematical literature. It generalizes classic situations. Therefore, its success continues to increase and to keep going up from time to time. In this work, we consider the model of Markov decision processes where the information on the costs includes imprecision. The fuzzy cost is represented by the fuzzy number set and the infinite horizon discounted cost is minimized from any stationary policy. This paper presents in the first part the notion of fuzzy sets and some axiomatic basis and relevant concepts with fuzzy theory in short. In second part, we propose a new definition of total discounted fuzzy cost in infinite planning horizon. We will compute an optimal stationary policy that minimizes the total fuzzy discounted cost by a new approach based on some standard algorithms of the dynamic programming using the ranking function concept. The last adapted criterion has many applications in several areas such that the forest management, the management of energy consumption, the finance, the communication system (mobile networks).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Archibald, T. W., & Possani, E. (2019). Investment and operational decisions for start-up companies: A game theory and Markov decision process approach. Annals of Operations Research,. https://doi.org/10.1007/s10479-019-03426-5.

    Article  Google Scholar 

  • Bahri, O., & Talbi, E. G. (2020). Robustness-based approach for fuzzy multi-objective problems. Annals of Operations Research,. https://doi.org/10.1007/s10479-020-03567-y.

    Article  Google Scholar 

  • Balbus, L., Jaśkiewicz, A., & Nowak, A. S. (2018). Markov perfect equilibria in a dynamic decision model with quasi-hyperbolic discounting. Annals of Operations Research, 287, 573–591.

    Article  Google Scholar 

  • Bellman, R. E., & Zadeh, L. A. (1970). Decision making in a fuzzy environment. Management Science, 17, 141–164.

    Article  Google Scholar 

  • Bertsekas, D. P., & Shreve, S. E. (1978). Stochastic optimal control. New York: Academic Press.

    Google Scholar 

  • Bhulai, S., Blok, H., & Spieksma, F. M. (2019). K competing queues with customer abandonment: optimality of a generalised \(\text{ c }\mu \)-rule by the Smoothed Rate Truncation method. Annals of Operations Research,. https://doi.org/10.1007/10479-019-03131-3.

    Article  Google Scholar 

  • Canbolat, P. G., & Rothblum, U. G. (2013). (Approximate) iterated successive approximations algorithm for sequential decision processes. Annals of Operations Research, 208, 309–320.

    Article  Google Scholar 

  • Derman, C. (1970). Finte state Markovian decision processes. New York: Academic Press.

    Google Scholar 

  • Diamond, P., & Kloeden, P. (1994). Metric spaces of fuzzy sets, theory and applications. Singapore: World Scientific.

    Book  Google Scholar 

  • Dubois, D., & Prade, H. (1983). Ranking of fuzzy numbers in the setting of possibility theory. Information Sciences, 30(3), 183–224.

    Article  Google Scholar 

  • Dubois, D., & Prade, H. (2000). Fundamentals of fuzzy sets. Boston: Kluwer Academic Publishers.

    Book  Google Scholar 

  • Howard, R. A. (1960). Dynamic programming and Markov processes. Cambridge: MIT Press.

    Google Scholar 

  • Buckley, J. J. (2005). Fuzzy probabilities. Berlin: Springer.

    Google Scholar 

  • Klir, G., & Yuan, B. (1995). Fuzzy sets and fuzzy logic, theory and applications. Upper Saddle River: Prentice Hall.

    Google Scholar 

  • Kurano, M., Nakagami, J., & Yoshida, Y. (2003). Markov decision processes with fuzzy rewards. Journal of Nonlinear Analysis and Convex Analysis, 4(1), 105–115.

    Google Scholar 

  • Mahdavi-Amiri, N., & Nasseri, S. H. (2006). Duality in fuzzy number linear programming by use of a certain linear ranking function. Applied Mathematics and Computation, 180, 206–216.

    Article  Google Scholar 

  • Mohammed, A. (2019). Towards a sustainable assessment of suppliers: An integrated fuzzy TOPSIS-possibilistic multi-objective approach. Annals of Operations Research,. https://doi.org/10.1007/s10479-019-03167-5.

    Article  Google Scholar 

  • Piunovskiy, A. B. (2013). Examples in Markov decision processes (Vol. 2). London: World Scientific.

    Google Scholar 

  • Powell, W. B. (2012). Perspectives of approximate dynamic programming. Annals of Operations Research, 13(2), 1–38.

    Google Scholar 

  • Puri, M. L., & Ralesca, D. A. (1986). Fuzzy random variable. Journal of Mathematical Analysis and Applications, 114, 402–422.

    Article  Google Scholar 

  • Puterman, M. L. (2014). Markov decision processes: Discrete stochastic dynamic programming. New York: Wiley.

    Google Scholar 

  • Roy, J., Pamuc̆ar, D., & Kar, S. (2019). Evaluation and selection of third party logistics provider under sustainability perspectives: An interval valued fuzzy-rough approach. Annals of Operations Research,. https://doi.org/10.1007/s10479-019-03501-x.

    Article  Google Scholar 

  • Yager, R. R. (1981). A procedure for ordering fuzzy subsets of the unit interval. Information Sciences, 24, 143–161.

    Article  Google Scholar 

  • Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8, 338–353.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Editor and the anonymous referees for their constructive comments, careful reading of the manuscript, valuable suggestions and of a number of helpful remarks which significantly improved the presentation of this paper. We also wish to express our sincere thanks to the following people. Firstly, Professor Dr. S. Melliani of Sultan Moulay Slimane University, Beni Mellal, Morocco for his help in fuzzy theory and encouragement during the period of research. Secondly, Mr. Lekbir Tansaoui, ELT teacher, co-author and textbook designer in Mokhtar Essoussi High School, Oued Zem, Morocco for proofreading this paper.

Author information

Authors and Affiliations

  1. Laboratory of Information Processing and Decision Support, Faculty of Sciences and Techniques, Sultan Moulay Slimane University, B.P. 523, Beni Mellal, Morocco

    Abdellatif Semmouri, Mostafa Jourhmane & Zineb Belhallaj

Authors
  1. Abdellatif Semmouri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mostafa Jourhmane
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zineb Belhallaj
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abdellatif Semmouri.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Semmouri, A., Jourhmane, M. & Belhallaj, Z. Discounted Markov decision processes with fuzzy costs. Ann Oper Res 295, 769–786 (2020). https://doi.org/10.1007/s10479-020-03783-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-020-03783-6

Keywords

Search

Navigation