Skip to main content
Springer Nature Link
Log in

Exploiting Robust Optimization for Interval Probabilistic Bisimulation

  • Conference paper
  • First Online:
Quantitative Evaluation of Systems (QEST 2016)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9826))

Included in the following conference series:

  • 1048 Accesses

Abstract

Verification of PCTL properties of MDPs with convex uncertainties has been investigated recently by Puggelli et al. However, model checking algorithms typically suffer from the state space explosion problem. In this paper, we discuss the use of probabilistic bisimulation to reduce the size of such an MDP while preserving the PCTL properties it satisfies. As a core part, we show that deciding bisimilarity of a pair of states can be encoded as adjustable robust counterpart of an uncertain LP. We show that using affine decision rules, probabilistic bisimulation relation can be approximated in polynomial time. We have implemented our approach and demonstrate its effectiveness on several case studies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Here, \(\mathcal {B}\) is the standard \(\sigma \)-algebra over \( Paths ^{\omega }\) generated from the set of all cylinder sets \(\{ Cyl _{\pi } \mid \pi \in Paths ^{*}\}\). The unique probability measure is obtained by the application of the extension theorem (see, e.g., [11]).

References

  1. Gurobi 4.0.2. http://www.gurobi.com/

  2. IBM ILOG CPLEX Optimizer. http://www.ibm.com/software/commerce/optimization/cplex-optimizer/

  3. PICOS: A Python interface for conic optimization solvers. http://picos.zib.de/

  4. PRISM model checker. http://www.prismmodelchecker.org/

  5. Abate, A., El Ghaoui, L.: Robust model predictive control through adjustable variables: an application to path planning. In: CDC, pp. 2485–2490 (2004)

    Google Scholar 

  6. Ben-Tal, A., El Ghaoui, L., Nemirovski, A.: Robust Optimization. Princeton University Press, Princeton (2009)

    Book  MATH  Google Scholar 

  7. Ben-Tal, A., Goryashko, A., Guslitzer, E., Nemirovski, A.: Adjustable robust solutions of uncertain linear programs. Math. Program. 99(2), 351–376 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  8. Ben-Tal, A., Nemirovski, A.: Robust solutions of uncertain linear programs. Oper. Res. Lett. 25, 1–13 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  9. Benedikt, M., Lenhardt, R., Worrell, J.: LTL model checking of interval Markov chains. In: Piterman, N., Smolka, S.A. (eds.) TACAS 2013 (ETAPS 2013). LNCS, vol. 7795, pp. 32–46. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  10. Bertsimas, D., Brown, D.B., Caramanis, C.: Theory and applications of robust optimization. SIAM Rev. 53(3), 464–501 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  11. Billingsley, P.: Probability and Measure. Wiley, New York (1979)

    MATH  Google Scholar 

  12. Cattani, S., Segala, R.: Decision algorithms for probabilistic bisimulation. In: Brim, L., Jančar, P., Křetínský, M., Kučera, A. (eds.) CONCUR 2002. LNCS, vol. 2421, pp. 371–385. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  13. Cattani, S., Segala, R., Kwiatkowska, M., Norman, G.: Stochastic transition systems for continuous state spaces and non-determinism. In: Sassone, V. (ed.) FOSSACS 2005. LNCS, vol. 3441, pp. 125–139. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  14. Chatterjee, K., Sen, K., Henzinger, T.A.: Model-checking \(\omega \)-regular properties of interval Markov chains. In: Amadio, R.M. (ed.) FOSSACS 2008. LNCS, vol. 4962, pp. 302–317. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  15. Chen, T., Han, T., Kwiatkowska, M.: On the complexity of model checking interval-valued discrete time Markov chains. Inf. Process. Lett. 113(7), 210–216 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  16. Coste, N., Hermanns, H., Lantreibecq, E., Serwe, W.: Towards performance prediction of compositional models in industrial GALS designs. In: Bouajjani, A., Maler, O. (eds.) CAV 2009. LNCS, vol. 5643, pp. 204–218. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  17. Dantzig, G.B., Madansky, A.: On the solution of two-stage linear programs under uncertainty. In: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1. pp. 165–176 (1961)

    Google Scholar 

  18. Delahaye, B., Katoen, J.P., Larsen, K.G., Legay, A., Pedersen, M.L., Sher, F., Wasowski, A.: New results on abstract probabilistic automata. In: ACSD, pp. 118–127 (2011)

    Google Scholar 

  19. Delahaye, B., Larsen, K.G., Legay, A., Pedersen, M.L., Wąsowski, A.: Decision problems for interval Markov chains. In: Dediu, A.-H., Inenaga, S., Martín-Vide, C. (eds.) LATA 2011. LNCS, vol. 6638, pp. 274–285. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  20. Fecher, H., Leucker, M., Wolf, V.: Don’t know in probabilistic systems. In: Valmari, A. (ed.) SPIN 2006. LNCS, vol. 3925, pp. 71–88. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  21. Ferrer Fioriti, L.M., Hashemi, V., Hermanns, H., Turrini, A.: Deciding probabilistic automata weak bisimulation: theory and practice. Form. Asp. Comput. 28(1), 109–143 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  22. Gebler, D., Hashemi, V., Turrini, A.: Computing behavioral relations for probabilistic concurrent systems. In: Remke, A., Stoelinga, M. (eds.) Stochastic Model Checking. LNCS, vol. 8453, pp. 117–155. Springer, Heidelberg (2014)

    Google Scholar 

  23. Givan, R., Leach, S.M., Dean, T.L.: Bounded-parameter Markov decision processes. Artif. Intell. 122(1–2), 71–109 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  24. Goerigk, M.: ROPI-a robust optimization programming interface for C++. Optim. Methods Softw. 29(6), 1261–1280 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  25. Grötschel, M., Lovász, L., Schrijver, A.: The ellipsoid method and its consequences in combinatorial optimization. Combinatorica 1(2), 169–197 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  26. Hahn, E.M., Han, T., Zhang, L.: Synthesis for PCTL in parametric Markov decision processes. In: Bobaru, M., Havelund, K., Holzmann, G.J., Joshi, R. (eds.) NFM 2011. LNCS, vol. 6617, pp. 146–161. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  27. Hashemi, V., Hatefi, H., Krčál, J.: Probabilistic bisimulations for PCTL model checking of interval MDPs (extended version). In: SynCoP. EPTCS, vol. 145, pp. 19–33 (2014)

    Google Scholar 

  28. Hashemi, V., Hermanns, H., Song, L., Subramani, K., Turrini, A., Wojciechowski, P.: Compositional bisimulation minimization for interval Markov decision processes. In: Dediu, A.-H., Janoušek, J., Martín-Vide, C., Truthe, B. (eds.) LATA 2016. LNCS, vol. 9618, pp. 114–126. Springer, Heidelberg (2016). doi:10.1007/978-3-319-30000-9_9

    Chapter  Google Scholar 

  29. Hermanns, H., Katoen, J.P.: Automated compositional Markov chain generation for a plain-old telephone system. Sci. Comput. Program. 36(1), 97–127 (2000)

    Article  MATH  Google Scholar 

  30. Iyengar, G.N.: Robust dynamic programming. Math. Oper. Res. 30(2), 257–280 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  31. Jonsson, B., Larsen, K.G.: Specification and refinement of probabilistic processes. In: LICS, pp. 266–277 (1991)

    Google Scholar 

  32. Kanellakis, P.C., Smolka, S.A.: CCS expressions, finite state processes, and three problems of equivalence. Inf. Comput. 86(1), 43–68 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  33. Karmarkar, N.: A new polynomial-time algorithm for linear programming. Combinatorica 4(4), 373–395 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  34. Khachyan, L.G.: A polynomial algorithm in linear programming. Sov. Math. Doklady 20(1), 191–194 (1979)

    Google Scholar 

  35. Kozine, I., Utkin, L.V.: Interval-valued finite Markov chains. Reliable Comput. 8(2), 97–113 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  36. Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  37. Löfberg, J.: Automatic robust convex programming. Optim. Methods Softw. 17(1), 115–129 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  38. Nilim, A., El Ghaoui, L.: Robust control of Markov decision processes with uncertain transition matrices. Oper. Res. 53(5), 780–798 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  39. Paige, R., Tarjan, R.E.: Three partition refinement algorithms. SIAM J. Comput. 16(6), 973–989 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  40. Puggelli, A.: Formal techniques for the verification and optimal control of probabilistic systems in the presence of modeling uncertainties. Ph.D. thesis, University of California, Berkeley (2014)

    Google Scholar 

  41. Puggelli, A., Li, W., Sangiovanni-Vincentelli, A.L., Seshia, S.A.: Polynomial-time verification of PCTL properties of MDPs with convex uncertainties. In: Sharygina, N., Veith, H. (eds.) CAV 2013. LNCS, vol. 8044, pp. 527–542. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  42. Segala, R.: Modeling and verification of randomized distributed real-time systems. Ph.D. thesis, MIT (1995)

    Google Scholar 

  43. Sen, K., Viswanathan, M., Agha, G.: Model-checking Markov chains in the presence of uncertainties. In: Hermanns, H., Palsberg, J. (eds.) TACAS 2006. LNCS, vol. 3920, pp. 394–410. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  44. Turrini, A., Hermanns, H.: Polynomial time decision algorithms for probabilistic automata. Inf. Comput. 244, 134–171 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  45. Wolff, E.M., Topcu, U., Murray, R.M.: Robust control of uncertain Markov decision processes with temporal logic specifications. In: CDC, pp. 3372–3379 (2012)

    Google Scholar 

  46. Wu, D., Koutsoukos, X.D.: Reachability analysis of uncertain systems using bounded-parameter Markov decision processes. Artif. Intell. 172(8–9), 945–954 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  47. Yi, W.: Algebraic reasoning for real-time probabilistic processes with uncertain information. Formal Techniques in Real-Time and Fault-Tolerant Systems. LNCS, vol. 863, pp. 680–693. Springer, Heidelberg (1994)

    Chapter  Google Scholar 

Download references

Acknowledgments

We would like to thank Arkadi Nemirovski (Georgia Institute of Technology) and Daniel Kuhn (EPFL) for many invaluable and insightful discussions. This work is supported by the EU 7th Framework Programme under grant agreements 295261 (MEALS) and 318490 (SENSATION), by the DFG as part of SFB/TR 14 AVACS, by the ERC Advanced Investigators Grant 695614 (POWVER), by the CAS/SAFEA International Partnership Program for Creative Research Teams, by the National Natural Science Foundation of China (Grants No. 61472473, 61532019, 61550110249, 61550110506), by the Chinese Academy of Sciences Fellowship for International Young Scientists, and by the CDZ project CAP (GZ 1023).

Author information

Authors and Affiliations

  1. State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, China

    Ernst Moritz Hahn & Andrea Turrini

  2. Max Planck Institute for Informatics, Saarbrücken, Germany

    Vahid Hashemi

  3. Department of Computer Science, Saarland University, Saarbrücken, Germany

    Vahid Hashemi & Holger Hermanns

Authors
  1. Ernst Moritz Hahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vahid Hashemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Holger Hermanns
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrea Turrini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vahid Hashemi .

Editor information

Editors and Affiliations

  1. University of Illinois , Urbana, Illinois, USA

    Gul Agha

  2. University of Antwerp , Antwerp, Belgium

    Benny Van Houdt

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Hahn, E.M., Hashemi, V., Hermanns, H., Turrini, A. (2016). Exploiting Robust Optimization for Interval Probabilistic Bisimulation. In: Agha, G., Van Houdt, B. (eds) Quantitative Evaluation of Systems. QEST 2016. Lecture Notes in Computer Science(), vol 9826. Springer, Cham. https://doi.org/10.1007/978-3-319-43425-4_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-43425-4_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-43424-7

  • Online ISBN: 978-3-319-43425-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics