Skip to main content
Springer Nature Link
Log in

Comprehensive Uncertainty Management in MDPs

  • Conference paper
Biologically Inspired Cognitive Architectures 2012

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 196))

  • 1426 Accesses

Abstract

Multistage decision-making in robots involved in real-world tasks is a process affected by uncertainty. The effects of the agent’s actions in a physical environment cannot be always predicted deterministically and in a precise manner. Moreover, observing the environment can be a too onerous for a robot, hence not continuos. Markov Decision Processes (MDPs) are a well-known solution inspired to the classic probabilistic approach for managing uncertainty. On the other hand, including fuzzy logics and possibility theory has widened uncertainty representation. Probability, possibility, fuzzy logics, and epistemic belief allow treating different and not always superimposable facets of uncertainty. This paper presents a new extended version of MDP, designed for managing all these kinds of uncertainty together to describe transitions between multi-valued fuzzy states. The motivation of this work is the design of robots that can be used to make decisions over time in an unpredictable environment. The model is described in detail along with its computational solution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Pardo, M.J., de la Fuente, D.: Design of a fuzzy finite capacity queuing model based on the degree of customer satisfaction: Analysis and fuzzy optimization. Fuzzy Sets and Syst. 159(24), 3313–3332 (2008)

    Article  MATH  Google Scholar 

  2. Klir, G.J., Parviz, B.: Probability-possibility transformations: A comparison. Int. J. Gen. Syst. 21(3), 291–310 (1992)

    Article  MATH  Google Scholar 

  3. Gwét, H.: Normalized conditional possibility distributions and informational connection between fuzzy variables. Int. J. Uncertain, Fuzziness and Know-Based Syst. 5(2), 177–198 (1997)

    Article  MATH  Google Scholar 

  4. Jumarie, G.: Possibility-probability transformation: A new result via information theory of deterministic functions. Kybernetes: Int. J. Syst. Cybernetics 23(5), 56–59 (1994)

    Article  Google Scholar 

  5. Sabbadin, R., Fargier, H., Lang, J.: Towards qualitative approaches to multi-stage decision making. Int. J. Approx. Reason. 19(3-4), 441–471 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  6. Zhou, Z.J., Hu, C.H., Zhang, B.C., Chen, L.: An improved fuzzy kalman filter for state estimation of nonlinear systems. J. Phys: Conference Series 96(1), 012130 (2008)

    Article  Google Scholar 

  7. Klir, G.: Principles of uncertainty: What are they? why do we need them? Fuzzy Sets Syst. 74(1), 15–31 (1995)

    Article  Google Scholar 

  8. Zadeh, L.A.: Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets Syst. 100(suppl. 1(0)), 9–34 (1999)

    Google Scholar 

  9. Matía, F., Jiménez, A., Al-Hadithi, B.M., Rodríguez-Losada, D., Galán, R.: The fuzzy kalman filter: State estimation using possibilistic techniques. Fuzzy Sets Syst. 157(16), 2145–2170 (2006)

    Article  MATH  Google Scholar 

  10. Wonneberger, S.: Generalization of an invertible mapping between probability and possibility. Fuzzy Sets Syst. 64, 229–240 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  11. Oussalah, M., De Schutter, J.: Possibilistic kalman filtering for radar 2d tracking. Inf. Sci. 130, 85–107 (2000)

    Article  MATH  Google Scholar 

  12. Dubois, D., Prade, H.: Unfair coins and necessity measures: Towards a possibilistic interpretation of histograms. Fuzzy Sets Syst. 10, 15–20 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  13. Chen, G., Xie, Q., Shieh, L.S.: Fuzzy kalman filtering. Inf. Sci. 109, 197–209 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  14. Klir, G., Folger, T.: Fuzzy sets, uncertainty, and information. Prentice Hall (1988)

    Google Scholar 

  15. Shafer, G.: A Mathematical Theory of Evidence. Princeton University Press, Princeton (1976)

    MATH  Google Scholar 

  16. Mohamed, M., Gader, P.: Generalized hidden markov models. part i. theoretical frameworks. IEEE Transactions on Fuzzy Syst. 8(1), 67–81 (2000)

    Article  MATH  Google Scholar 

  17. Giang, P.H., Shenoy, P.P.: A qualitative linear utility theory for Spohns theory of epistemic beliefs. In: UAI, pp. 220–229. Morgan Kaufmann (2000)

    Google Scholar 

  18. Dubois, D., Prade, H.: Possibility theory as a basis for qualitative decision theory. In: Proceedings of the 14th International Joint Conference on Artificial Intelligence, vol. 2, pp. 1924–1930. Morgan Kaufmann Publishers Inc., San Francisco (1995)

    Google Scholar 

  19. Ghorbani, A.A., Xu, X.: A fuzzy markov model approach for predicting user navigation. In: IEEE/WIC/ACM International Conference on Web Intelligence, pp. 307–311 (2007)

    Google Scholar 

  20. Yamada, K.: Probability-possibility transformation based on evidence theory. In: Joint 9th IFSA World Congress and 20th NAFIPS International Conference, vol. 1, pp. 70–75 (2001)

    Google Scholar 

  21. Hsu, K., Marcus, S.I.: Decentralized control of finite state Markov processes. IEEE Trans. Auto. Control AC-27(2), 426–431 (1982)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Ingegneria Chimica, Gestionale, Informatica, Meccanica, University of Palermo, Viale delle Scienze, Edificio 6, Palermo, Italy

    Vincenzo Cannella, Roberto Pirrone & Antonio Chella

Authors
  1. Vincenzo Cannella
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roberto Pirrone
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antonio Chella
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vincenzo Cannella .

Editor information

Editors and Affiliations

  1. Department of Chemical, Management,, Computer, Mechanical Engineering, Università di Palermo, Viale delle Scienze, Building 6 6, Palermo, 90128, Italy

    Antonio Chella

  2. Department of Chemical, Management,, Computer, Mechanical Engineering, Università di Palermo, Viale delle Scienze, Building 6 6, Palermo, 90128, Italy

    Roberto Pirrone

  3. Department of Chemical, Management,, Computer, Mechanical Engineering, Università di Palermo, Viale delle Scienze, Building 6 6, Palermo, 90128, Italy

    Rosario Sorbello

  4. , Department of Psychology, Reykjavik University, Menntavegur 1, Reykjavik, 101, Iceland

    Kamilla Rún Jóhannsdóttir

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Cannella, V., Pirrone, R., Chella, A. (2013). Comprehensive Uncertainty Management in MDPs. In: Chella, A., Pirrone, R., Sorbello, R., Jóhannsdóttir, K. (eds) Biologically Inspired Cognitive Architectures 2012. Advances in Intelligent Systems and Computing, vol 196. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34274-5_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-34274-5_20

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-34273-8

  • Online ISBN: 978-3-642-34274-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics