Skip to main content
SpringerLink
Log in

Fuzzy Markov Chains and Decision-Making

  • Published:
Fuzzy Optimization and Decision Making Aims and scope Submit manuscript

Abstract

Decision-making in an environment of uncertainty and imprecision for real-world problems is a complex task. In this paper it is introduced general finite state fuzzy Markov chains that have a finite convergence to a stationary (may be periodic) solution. The Cesaro average and the α-potential for fuzzy Markov chains are defined, then it is shown that the relationship between them corresponds to the Blackwell formula in the classical theory of Markov decision processes. Furthermore, it is pointed out that recurrency does not necessarily imply ergodicity. However, if a fuzzy Markov chain is ergodic, then the rows of its ergodic projection equal the greatest eigen fuzzy set of the transition matrix. Then, the fuzzy Markov chain is shown to be a robust system with respect to small perturbations of the transition matrix, which is not the case for the classical probabilistic Markov chains. Fuzzy Markov decision processes are finally introduced and discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abbad, M. and J.A. Filar. (1992). “Perturbation and Stability Theory for Markov Control Problems,” IEEE Trans. Auto. Contr. 37, 1415–1420.

    Google Scholar 

  • Avrachenkov, K.E. and E. Sanchez. (2000). “Fuzzy Markov Chains: Specificities and Properties,” Proceedings of the 8-th IPMU'2000 Conference, Madrid,Spain, 1851–1856.

  • Bellman, R.E. (1957). Dynamic Programming, Princeton University Press.

  • Bellman, R.E. and L.A. Zadeh. (1970). “Decision-Making in a Fuzzy Environment,” Management Science Series B 17, 141–164.

    Google Scholar 

  • Chung, K.L. (1960). Markov Chains with Stationary Transition Probabilities, New York: Springer-Verlag.

    Google Scholar 

  • Delebecque, F. and J.P. Quadrat. (1981). “Optimal Control of Markov Chains Admitting Strong and Weak Interactions,” Automatica 17, 281–296.

    Google Scholar 

  • Doob, J.L. (1953). Stochastic Processes, New York: John Wiley.

    Google Scholar 

  • Esogbue, A.O. and R.E. Bellman. (1984). “Fuzzy Dynamic Programming and its Extensions,” TIMS/Studies in the Management Sciences 20, 147–167.

    Google Scholar 

  • Give'on, Y. (1964). “Lattice Matrices,” Information and Control 7, 477–484. Kruse, R., R. Buck-Emden, and R. Cordes. (1987). “Processor Power Considerations-An Application of Fuzzy Markov Chains,” Fuzzy Sets and Systems 21, 289-299.

    Google Scholar 

  • Kurano, M., M. Yasuda, J. Nakagami, and Y. Yoshida. (1992). “A Limit Theorem in Some Fuzzy Dynamic Systems,” Fuzzy Sets and Systems 51, 83–88.

    Google Scholar 

  • Kurano, M., M. Yasuda, J. Nakagami, and Y. Yoshida. (1996). “Markov-Type Fuzzy Decision Processes with a Discounted Reward on a Closed Interval,” European Journal of Operational Research 92, 649–662.

    Google Scholar 

  • Pervozvanskii, A.A. and V.G. Gaitsgori. (1988). Theory of Suboptimal Decisions, Dordrecht: Kluwer Academic Publishers.

    Google Scholar 

  • Puterman, M.L. (1994). Markov Decision Processes, New York: John Wiley.

    Google Scholar 

  • Sanchez, E. (1976). “Resolution of Composite Fuzzy Relation Equations,” Information and Control 30, 38–48.

    Google Scholar 

  • Sanchez, E. (1978). “Resolution of Eigen Fuzzy Sets Equations,” Fuzzy Sets and Systems 1, 69–74.

    Google Scholar 

  • Sanchez, E. (1981). “Eigen Fuzzy Sets and Fuzzy Relations,” J. Math. Anal. Appl. 81, 399–421.

    Google Scholar 

  • Schweizer, B. and A. Sklar. (1963). “Associative Functions and Abstract Semigroups,” Publ. Math. Debrecen. 10, 69–81.

    Google Scholar 

  • Schweitzer, P.J. (1968). “Perturbation Theory and Finite Markov Chains,” J. App. Prob. 5, 401–413.

    Google Scholar 

  • Thomason, M.G. (1977). “Convergence of Powers of a Fuzzy Matrix,” J. Math. Anal. Appl. 57, 476–480.

    Google Scholar 

  • Yoshida, Y., M. Yasuda, J. Nakagami, and M. Kurano. (1993). “A Potential of Fuzzy Relations with a Linear Structure: The Contractive Case,” Fuzzy Sets and Systems 60, 283–294.

    Google Scholar 

  • Yoshida, Y., M. Yasuda, J. Nakagami, and M. Kurano. (1994). “A Potential of Fuzzy Relations with a Linear Structure: The Unbounded Case,” Fuzzy Sets and Systems 66, 83–95.

    Google Scholar 

  • Yoshida, Y. (1994). “Markov Chains with a Transition Possibility Measure and Fuzzy Dynamic Programming,” Fuzzy Sets and Systems 66, 39–57.

    Google Scholar 

  • Zadeh, L.A. (1965). “Fuzzy Sets,” Information and Control 8, 338–353.

    Google Scholar 

  • Zadeh, L.A. (1998). “Maximizing Sets and Fuzzy Markoff Algorithms,” IEEE Trans. on Systems, Man, and Cybernetics-Part C: Applications and Reviews 28, 9–15.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. INRIA Sophia Antipolis, 2004 route des Lucioles, B.P.93, 06902, France

    Konstantin E. Avrachenkov

  2. Laboratoire d'Informatique Médicale, Faculté de Médecine, 27 Bd Jean Moulin, 13385, Marseille Cedex 5, France

    Elie Sanchez

Authors
  1. Konstantin E. Avrachenkov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elie Sanchez
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Avrachenkov, K.E., Sanchez, E. Fuzzy Markov Chains and Decision-Making. Fuzzy Optimization and Decision Making 1, 143–159 (2002). https://doi.org/10.1023/A:1015729400380

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1015729400380

Search

Navigation