Skip to main content

A Recursive Algorithm for Computing Inferences in Imprecise Markov Chains

  • Conference paper
  • First Online:
Symbolic and Quantitative Approaches to Reasoning with Uncertainty (ECSQARU 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11726))

  • 600 Accesses

Abstract

We present an algorithm that can efficiently compute a broad class of inferences for discrete-time imprecise Markov chains, a generalised type of Markov chains that allows one to take into account partially specified probabilities and other types of model uncertainty. The class of inferences that we consider contains, as special cases, tight lower and upper bounds on expected hitting times, on hitting probabilities and on expectations of functions that are a sum or product of simpler ones. Our algorithm exploits the specific structure that is inherent in all these inferences: they admit a general recursive decomposition. This allows us to achieve a computational complexity that scales linearly in the number of time points on which the inference depends, instead of the exponential scaling that is typical for a naive approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Within the field of imprecise probability theory, this model is called an imprecise Markov chain under epistemic irrelevance [5, 6, 9].

References

  1. De Bock, J.: Credal networks under epistemic irrelevance. Int. J. Approximate Reasoning 85, 107–138 (2017)

    Article  MathSciNet  Google Scholar 

  2. De Bock, J.: Credal networks under epistemic irrelevance: theory and algorithms. Ph.D. thesis, Ghent University (2015)

    Google Scholar 

  3. de Cooman, G., Hermans, F.: Imprecise probability trees: bridging two theories of imprecise probability. Artif. Intell. 172(11), 1400–1427 (2008)

    Article  MathSciNet  Google Scholar 

  4. de Cooman, G., Hermans, F., Antonucci, A., Zaffalon, M.: Epistemic irrelevance in credal nets: the case of imprecise Markov trees. Int. J. Approximate Reasoning 51(9), 1029–1052 (2010)

    Article  MathSciNet  Google Scholar 

  5. de Cooman, G., Hermans, F., Quaeghebeur, E.: Imprecise Markov chains and their limit behaviour. Probab. Eng. Inf. Sci. 23(4), 597–635 (2009)

    Article  Google Scholar 

  6. Hermans, F., Škulj, D.: Stochastic processes. In: Augustin, T., Coolen, F.P.A., De Cooman, G., Troffaes, M.C.M. (eds.) Introduction to Imprecise Probabilities. Wiley (2014). Chapter 11

    Google Scholar 

  7. Kallenberg, O.: Foundations of Modern Probability. Springer Science & Business Media, New York (2006)

    MATH  Google Scholar 

  8. Krak, T’Joens, N., De Bock, J.: Hitting times and probabilities for imprecise Markov chains (2019). Accepted for publication in the conference proceedings of ISIPTA 2019. A preprint can be found at arXiv:1905.08781

  9. Lopatatzidis, S.: Robust Modelling and Optimisation in Stochastic Processes using Imprecise Probabilities, with an Application to Queueing Theory. Ph.D. thesis, Ghent University (2017)

    Google Scholar 

  10. Shafer, G., Vovk, V.: Probability and Finance: It’s Only a Game!. Wiley, New York (2001)

    Book  Google Scholar 

  11. T’Joens, N., De Bock, J., de Cooman, G.: In search of a global belief model for discrete-time uncertain processes (2019). Accepted for publication in the conference proceedings of ISIPTA 2019

    Google Scholar 

  12. T’Joens, N., De Bock, J., de Cooman, G.: Continuity properties of game-theoretic upper expectations. arXiv:1902.09406 (2019)

  13. T’Joens, N., Krak, T., De Bock, J., de Cooman, G.: A Recursive Algorithm for Computing Inferences in Imprecise Markov Chains. arXiv:1905.12968 (2019)

Download references

Acknowledgments

The work in this paper was partially supported by H2020-MSCA-ITN-2016 UTOPIAE, grant agreement 722734.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Natan T’Joens .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

T’Joens, N., Krak, T., Bock, J.D., Cooman, G.d. (2019). A Recursive Algorithm for Computing Inferences in Imprecise Markov Chains. In: Kern-Isberner, G., Ognjanović, Z. (eds) Symbolic and Quantitative Approaches to Reasoning with Uncertainty. ECSQARU 2019. Lecture Notes in Computer Science(), vol 11726. Springer, Cham. https://doi.org/10.1007/978-3-030-29765-7_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-29765-7_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-29764-0

  • Online ISBN: 978-3-030-29765-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics