Skip to main content
Springer Nature Link
Log in

Compositional Bisimulation Minimization for Interval Markov Decision Processes

  • Conference paper
  • First Online:
Language and Automata Theory and Applications (LATA 2016)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9618))

Included in the following conference series:

  • 1582 Accesses

Abstract

Formal verification of PCTL properties of MDPs with convex uncertainties has been recently investigated by Puggelli et al. However, model checking algorithms typically suffer from state space explosion. In this paper, we address probabilistic bisimulation to reduce the size of such an MDP while preserving PCTL properties it satisfies. We give a compositional reasoning over interval models to understand better the ways how large models with interval uncertainties can be composed. Afterwards, we discuss computational complexity of the bisimulation minimization and show that the problem is coNP-complete. Finally, we show that, under a mild condition, bisimulation can be computed in polynomial time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Here, \(\mathcal {B}\) is the standard \(\sigma \)-algebra over \( Paths _{ inf }\) generated from the set of all cylinder sets \(\{ Paths _{\omega } \mid \omega \in Paths _{ fin }\}\). The unique probability measure is obtained by the application of the extension theorem (see, e.g. [3]).

References

  1. Balas, E.: Disjunctive programming: properties of the convex hull of feasible points. Discrete Appl. Math. 89(1), 3–44 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  2. Benedikt, M., Lenhardt, R., Worrell, J.: LTL model checking of interval Markov chains. In: Piterman, N., Smolka, S.A. (eds.) TACAS 2013 (ETAPS 2013). LNCS, vol. 7795, pp. 32–46. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  3. Billingsley, P.: Probability and Measure. John Wiley and Sons, New York (1979)

    MATH  Google Scholar 

  4. Cattani, S., Segala, R.: Decision algorithms for probabilistic bisimulation. In: Brim, L., Jančar, P., Křetínský, M., Kučera, A. (eds.) CONCUR 2002. LNCS, vol. 2421, pp. 371–385. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  5. Cattani, S., Segala, R., Kwiatkowska, M., Norman, G.: Stochastic transition systems for continuous state spaces and non-determinism. In: Sassone, V. (ed.) FOSSACS 2005. LNCS, vol. 3441, pp. 125–139. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  6. Chatterjee, K., Sen, K., Henzinger, T.A.: Model-checking \(\omega \)-regular properties of interval Markov chains. In: Amadio, R.M. (ed.) FOSSACS 2008. LNCS, vol. 4962, pp. 302–317. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  7. Chehaibar, G., Garavel, H., Mounier, L., Tawbi, N., Zulian, F.: Specification and verification of the PowerScale\({}^{\textregistered }\) busarbitration protocol: an industrial experiment with LOTOS. In: FORTE, pp. 435–450 (1996)

    Google Scholar 

  8. Chen, T., Han, T., Kwiatkowska, M.: On the complexity of model checking interval-valued discrete time Markov chains. Inf. Process. Lett. 113(7), 210–216 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  9. Coste, N., Hermanns, H., Lantreibecq, E., Serwe, W.: Towards performance prediction of compositional models in industrial GALS designs. In: Bouajjani, A., Maler, O. (eds.) CAV 2009. LNCS, vol. 5643, pp. 204–218. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  10. Delahaye, B., Katoen, J.-P., Larsen, K.G., Legay, A., Pedersen, M.L., Sher, F., Wąsowski, A.: Abstract probabilistic automata. In: Jhala, R., Schmidt, D. (eds.) VMCAI 2011. LNCS, vol. 6538, pp. 324–339. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  11. Delahaye, B., Larsen, K.G., Legay, A., Pedersen, M.L., Wąsowski, A.: Decision problems for interval Markov chains. In: Dediu, A.-H., Inenaga, S., Martín-Vide, C. (eds.) LATA 2011. LNCS, vol. 6638, pp. 274–285. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  12. Eirinakis, P., Ruggieri, S., Subramani, K., Wojciechowski, P.: On quantified linear implications. AMAI 71(4), 301–325 (2014)

    MathSciNet  MATH  Google Scholar 

  13. Garey, M.R., Johnson, D.S.: Computers and Intractability; A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., New York (1990)

    Google Scholar 

  14. Hahn, E.M., Han, T., Zhang, L.: Synthesis for PCTL in parametric Markov decision processes. In: Bobaru, M., Havelund, K., Holzmann, G.J., Joshi, R. (eds.) NFM 2011. LNCS, vol. 6617, pp. 146–161. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  15. Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Formal Aspects Comput. 6(5), 512–535 (1994)

    Article  MATH  Google Scholar 

  16. Hashemi, V., Hatefi, H., Krčál, J.: Probabilistic bisimulations for PCTL model checking of interval MDPs. SynCoP EPTCS 145, 19–33 (2014)

    Article  Google Scholar 

  17. Hashemi, V., Hermanns, H., Turrini, A.: On the efficiency of deciding probabilistic automata weak bisimulation. ECEASST, 66 (2013)

    Google Scholar 

  18. Hermanns, H., Katoen, J.-P.: Automated compositional Markov chain generation for a plain-old telephone system. Sci. Comp. Progr. 36(1), 97–127 (2000)

    Article  MATH  Google Scholar 

  19. Iancu, D.A., Sharma, M., Sviridenko, M.: Supermodularity and affine policies in dynamic robust optimization. Oper. Res. 61(4), 941–956 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  20. Iyengar, G.N.: Robust dynamic programming. Math. Oper. Res. 30(2), 257–280 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  21. Jonsson, B., Larsen, K.G.: Specification and refinement of probabilistic processes. In: LICS, pp. 266–277 (1991)

    Google Scholar 

  22. Kanellakis, P.C., Smolka, S.A.: CCS expressions, finite state processes, and three problems of equivalence. Inf. Comput. 86(1), 43–68 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  23. Katoen, J.-P., Kemna, T., Zapreev, I., Jansen, D.N.: Bisimulation minimisation mostly speeds up probabilistic model checking. In: Grumberg, O., Huth, M. (eds.) TACAS 2007. LNCS, vol. 4424, pp. 87–101. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  24. Katoen, J.-P., Klink, D., Neuhäußer, M.R.: Compositional abstraction for stochastic systems. In: Ouaknine, J., Vaandrager, F.W. (eds.) FORMATS 2009. LNCS, vol. 5813, pp. 195–211. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  25. Kozine, I., Utkin, L.V.: Interval-valued finite Markov chains. Reliable Comput. 8(2), 97–113 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  26. Kroening, D., Strichman, O.: Decision Procedures: An Algorithmic Point of View, 1st edn. Springer, Heidelberg (2008)

    Google Scholar 

  27. Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  28. Nilim, A., Ghaoui, L.E.: Robust control of Markov decision processes with uncertain transition matrices. Oper. Res. 53(5), 780–798 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  29. Paige, R., Tarjan, R.E.: Three partition refinement algorithms. SIAM J. Comput. 16(6), 973–989 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  30. Puggelli, A., Li, W., Sangiovanni-Vincentelli, A.L., Seshia, S.A.: Polynomial-time verification of PCTL properties of MDPs with convex uncertainties. In: Sharygina, N., Veith, H. (eds.) CAV 2013. LNCS, vol. 8044, pp. 527–542. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  31. Schrijver, A.: Theory of Linear and Integer Programming. John Wiley & Sons, Amsterdam (1998)

    MATH  Google Scholar 

  32. Segala, R.: Modeling and verification of randomized distributed real-time systems. Ph.D. thesis, MIT (1995)

    Google Scholar 

  33. Sen, K., Viswanathan, M., Agha, G.: Model-checking Markov chains in the presence of uncertainties. In: Hermanns, H., Palsberg, J. (eds.) TACAS 2006. LNCS, vol. 3920, pp. 394–410. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  34. Subramani, K.: On the complexities of selected satisfiability and equivalence queries over boolean formulas and inclusion queries over hulls. JAMDS, 2009 (2009)

    Google Scholar 

  35. Turrini, A., Hermanns, H.: Polynomial time decision algorithms for probabilistic automata. Inf. Comput. 244, 134–171 (2015)

    Article  MathSciNet  Google Scholar 

  36. Vaidya, P.M.: An algorithm for linear programming which requires \({O}(((m+n)n^2) + (m+n)^{1.5})n){L})\) arithmetic operations. Math. Program. 47, 175–201 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  37. Wolff, E.M., Topcu, U., Murray, R.M.: Robust control of uncertain Markov decision processes with temporal logic specifications. In: CDC, pp. 3372–3379 (2012)

    Google Scholar 

  38. Wu, D., Koutsoukos, X.D.: Reachability analysis of uncertain systems using bounded-parameter Markov decision processes. AI 172(8–9), 945–954 (2008)

    MathSciNet  MATH  Google Scholar 

  39. Yi, W.: Algebraic reasoning for real-time probabilistic processes with uncertain information. In: Langmaack, H., de Roever, W.-P., Vytopil, J. (eds.) FTRTFT 1994 and ProCoS 1994. LNCS, vol. 863, pp. 680–693. Springer, Heidelberg (1994)

    Chapter  Google Scholar 

Download references

Acknowledgments

This work is supported by the EU 7th Framework Programme under grant agreements 295261 (MEALS) and 318490 (SENSATION), by the DFG as part of SFB/TR 14 AVACS, by the CAS/SAFEA International Partnership Program for Creative Research Teams, by the National Natural Science Foundation of China (Grants 61472473 and 61550110249), by the Chinese Academy of Sciences Fellowship for International Young Scientists (Grant 2015VTC029), and by the CDZ project CAP (GZ 1023). This research is supported in part by the National Science Foundation through Award CCF-1305054.

Author information

Authors and Affiliations

  1. Max Planck Institute for Informatics, Saarbrücken, Germany

    Vahid Hashemi

  2. Department of Computer Science, Saarland University, Saarbrücken, Germany

    Vahid Hashemi & Holger Hermanns

  3. University of Technology Sydney, Sydney, Australia

    Lei Song

  4. State Key Laboratory of Computer Science, Institute of Software, CAS, Beijing, China

    Lei Song & Andrea Turrini

  5. LDCSEE, West Virginia University, Morgantown, WV, USA

    K. Subramani & Piotr Wojciechowski

Authors
  1. Vahid Hashemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Holger Hermanns
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lei Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. K. Subramani
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Andrea Turrini
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Piotr Wojciechowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vahid Hashemi .

Editor information

Editors and Affiliations

  1. Rovira i Virgili University, Tarragona, Spain

    Adrian-Horia Dediu

  2. Czech Technical University, Prague, Czech Republic

    Jan Janoušek

  3. Rovira i Virgili University, Tarragona, Spain

    Carlos Martín-Vide

  4. Justus Liebig University Giessen, Gießen, Germany

    Bianca Truthe

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Hashemi, V., Hermanns, H., Song, L., Subramani, K., Turrini, A., Wojciechowski, P. (2016). Compositional Bisimulation Minimization for Interval Markov Decision Processes. In: Dediu, AH., Janoušek, J., Martín-Vide, C., Truthe, B. (eds) Language and Automata Theory and Applications. LATA 2016. Lecture Notes in Computer Science(), vol 9618. Springer, Cham. https://doi.org/10.1007/978-3-319-30000-9_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-30000-9_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-29999-0

  • Online ISBN: 978-3-319-30000-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics