Skip to main content
SpringerLink
Log in

Modal Stochastic Games

Abstraction-Refinement of Probabilistic Automata

  • Chapter
  • First Online:
Models, Algorithms, Logics and Tools

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10460))

Abstract

This paper presents an abstraction-refinement framework for Segala’s probabilistic automata (PA), a slight variant of Markov decision processes. We use Condon and Ladner’s two-player probabilistic game automata extended with possible and required transitions—as in Larsen and Thomsen’s modal transition systems—as abstract models. The key idea is to refine player-one and player-two states separately resulting in a nested abstract-refine loop. We show the adequacy of this approach for obtaining tight bounds on extremal reachability probabilities.

This work has been partially funded by the Excellence Initiative of the German federal and state government and the CDZ project CAP (GZ 1023).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    As this paper does not cover parallel composition all PGAs are closed. For modeling PGAs in a compositonal manner though, the distinction between internal and other actions is important, see [7].

  2. 2.

    For example, let \(x=\max \) in \( \mathrm {Pr}^x(T^{\prime })\) then \(\mathbf {1}=\max \) and \(\mathbf {2}=\min \) (player-one maximizes whereas the player-two minimizes the probability) or vice versa.

  3. 3.

    This may converge slower than allowing for coarser splittings (as in [5]), but yields smaller state spaces.

References

  1. Segala, R., Lynch, N.A.: Probabilistic simulations for probabilistic processes. Nordic J. Comput. 2(2), 250–273 (1995)

    MathSciNet  MATH  Google Scholar 

  2. Norman, G.: Analysing randomized distributed algorithms. In: Baier, C., Haverkort, B.R., Hermanns, H., Katoen, J.-P., Siegle, M. (eds.) Validation of Stochastic Systems. LNCS, vol. 2925, pp. 384–418. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24611-4_11

    Chapter  Google Scholar 

  3. Huth, M.: On finite-state approximants for probabilistic computation tree logic. Theoret. Comput. Sci. 346(1), 113–134 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  4. Delahaye, B., Katoen, J.P., Larsen, K.G., Legay, A., Pedersen, M.L., Sher, F., Wasowski, A.: Abstract probabilistic automata. Inf. Comput. 232, 66–116 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  5. Kattenbelt, M., Kwiatkowska, M.Z., Norman, G., Parker, D.: A game-based abstraction-refinement framework for Markov decision processes. Formal Methods Syst. Des. 36(3), 246–280 (2010)

    Article  MATH  Google Scholar 

  6. Vira, F.S., Katoen, J.-P.: Tight game abstractions of probabilistic automata. In: Baldan, P., Gorla, D. (eds.) CONCUR 2014. LNCS, vol. 8704, pp. 576–591. Springer, Heidelberg (2014). doi:10.1007/978-3-662-44584-6_39

    Google Scholar 

  7. Sher, F., Katoen, J.-P.: Compositional abstraction techniques for probabilistic automata. In: Baeten, J.C.M., Ball, T., Boer, F.S. (eds.) TCS 2012. LNCS, vol. 7604, pp. 325–341. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33475-7_23

    Chapter  Google Scholar 

  8. Condon, A., Ladner, R.E.: Probabilistic game automata. J. Comput. Syst. Sci. 36(3), 452–489 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  9. Antonik, A., Huth, M., Larsen, K.G., Nyman, U., Wasowski, A.: 20 years of modal and mixed specifications. Bull. EATCS 95, 94–129 (2008)

    MathSciNet  MATH  Google Scholar 

  10. Huth, M., Jagadeesan, R., Schmidt, D.: Modal transition systems: a foundation for three-valued program analysis. In: Sands, D. (ed.) ESOP 2001. LNCS, vol. 2028, pp. 155–169. Springer, Heidelberg (2001). doi:10.1007/3-540-45309-1_11

    Chapter  Google Scholar 

  11. Shapley, L.S.: Stochastic games. Proc. Natl. Acad. Sci. USA 39(10), 1095–1100 (1953)

    Article  MathSciNet  MATH  Google Scholar 

  12. Baier, C., Katoen, J.P.: Principles of Model Checking. MIT Press, Cambridge (2008)

    MATH  Google Scholar 

  13. Bertsekas, D.P., Tsitsiklis, J.N.: An analysis of stochastic shortest path problems. Math. Oper. Res. 16, 580–595 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  14. Tarski, A.: A lattice-theoretical fixpoint theorem and its applications. Pacific J. of Math. 5(2), 285–309 (1955)

    Article  MathSciNet  MATH  Google Scholar 

  15. Baier, C., Engelen, B., Majster-Cederbaum, M.E.: Deciding bisimilarity and similarity for probabilistic processes. J. Comput. Syst. Sci. 60(1), 187–231 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  16. Jonsson, B., Larsen, K.G.: Specification and refinement of probabilistic processes. In: LICS, pp. 266–277. IEEE Computer Society (1991)

    Google Scholar 

  17. Larsen, K.G., Thomsen, B.: Compositional proofs by partial specification of processes. In: Chytil, M.P., Koubek, V., Janiga, L. (eds.) MFCS 1988. LNCS, vol. 324, pp. 414–423. Springer, Heidelberg (1988). doi:10.1007/BFb0017164

    Chapter  Google Scholar 

  18. Sher, F.: Abstraction and refinement of probabilistic automata using modal stochastic games. Ph.D. thesis, RWTH Aachen University Aachener Informatik-Berichte AIB-2015-10 (2015)

    Google Scholar 

  19. D’Argenio, P.R., Jeannet, B., Jensen, H.E., Larsen, K.G.: Reachability analysis of probabilistic systems by successive refinements. In: Alfaro, L., Gilmore, S. (eds.) PAPM-PROBMIV 2001. LNCS, vol. 2165, pp. 39–56. Springer, Heidelberg (2001). doi:10.1007/3-540-44804-7_3

    Chapter  Google Scholar 

  20. Katoen, J.P., Klink, D., Leucker, M., Wolf, V.: Three-valued abstraction for probabilistic systems. J. Log. Algebr. Program. 81(4), 356–389 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  21. Hermanns, H., Wachter, B., Zhang, L.: Probabilistic CEGAR. In: Gupta, A., Malik, S. (eds.) CAV 2008. LNCS, vol. 5123, pp. 162–175. Springer, Heidelberg (2008). doi:10.1007/978-3-540-70545-1_16

    Chapter  Google Scholar 

  22. Dehnert, C., Gebler, D., Volpato, M., Jansen, D.N.: On abstraction of probabilistic systems. In: Remke, A., Stoelinga, M. (eds.) Stochastic Model Checking. Rigorous Dependability Analysis Using Model Checking Techniques for Stochastic Systems. LNCS, vol. 8453, pp. 87–116. Springer, Heidelberg (2014). doi:10.1007/978-3-662-45489-3_4

    Chapter  Google Scholar 

  23. de Alfaro, L., Godefroid, P., Jagadeesan, R.: Three-valued abstractions of games: uncertainty, but with precision. In: LICS, pp. 170–179. IEEE Computer Society (2004)

    Google Scholar 

  24. Cousot, P., Monerau, M.: Probabilistic abstract interpretation. In: Seidl, H. (ed.) ESOP 2012. LNCS, vol. 7211, pp. 169–193. Springer, Heidelberg (2012). doi:10.1007/978-3-642-28869-2_9

    Chapter  Google Scholar 

  25. Wachter, B., Zhang, L.: Best probabilistic transformers. In: Barthe, G., Hermenegildo, M. (eds.) VMCAI 2010. LNCS, vol. 5944, pp. 362–379. Springer, Heidelberg (2010). doi:10.1007/978-3-642-11319-2_26

    Chapter  Google Scholar 

  26. Hermanns, H., Krčál, J., Křetínský, J.: Probabilistic bisimulation: naturally on distributions. In: Baldan, P., Gorla, D. (eds.) CONCUR 2014. LNCS, vol. 8704, pp. 249–265. Springer, Heidelberg (2014). doi:10.1007/978-3-662-44584-6_18

    Google Scholar 

  27. Larsen, K.G., Skou, A.: Bisimulation through probabilistic testing. Inf. Comput. 94(1), 1–28 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  28. Larsen, K.G., Thomsen, B.: A modal process logic. In: LICS, pp. 203–210. IEEE Computer Society (1988)

    Google Scholar 

  29. Caillaud, B., Delahaye, B., Larsen, K.G., Legay, A., Pedersen, M.L., Wasowski, A.: Constraint Markov chains. Theoret. Comput. Sci. 412(34), 4373–4404 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  30. Bengtsson, J., Larsen, K.G., Larsson, F., Pettersson, P., Yi, W.: Uppaal in 1995. In: Margaria, T., Steffen, B. (eds.) TACAS 1996. LNCS, vol. 1055, pp. 431–434. Springer, Heidelberg (1996). doi:10.1007/3-540-61042-1_66

    Chapter  Google Scholar 

  31. D’Argenio, P.R., Katoen, J.-P., Ruys, T.C., Tretmans, J.: The bounded retransmission protocol must be on time!. In: Brinksma, E. (ed.) TACAS 1997. LNCS, vol. 1217, pp. 416–431. Springer, Heidelberg (1997). doi:10.1007/BFb0035403

    Chapter  Google Scholar 

Download references

Acknowledgements

This work is strongly inspired by and heavily builds upon the work of Kim G. Larsen. The idea of using possible (may) and required (must) transitions goes back to his seminal work with Thomsen [28]. Simulation and refinement relations for probabilistic models originated in his work with Jonsson [16]. Kim developed one of the first, if not the very first, abstraction-refinement technique for MDPs [19]. His work on constraint Markov chains [29] provided the basis for our joint work on abstract PAs [4]. The uncertainty of the non-deterministic choices in APA is modeled by modal transitions while uncertainty of the stochastic behavior is expressed—as in constraint Markov chains—by (underspecified) stochastic constraints. Besides the influence of all these work, Kim has always been extremely inspiring. This started in 1996 at the conference FTRTFT in Uppsala, when he stimulated us to use Uppaal—at those days in its very early stage of development [30]—to take up the challenge of modeling and verifying Philips’ bounded retransmission protocol [31]. This relationship has continued over the years and has led to several joint EU projects. It has been a great pleasure and enormous honor to work with Kim. This paper is a salute to his 60th birthday.

Author information

Authors and Affiliations

  1. RWTH Aachen University, Aachen, Germany

    Joost-Pieter Katoen

  2. Information Technology University, Lahore, Punjab, Pakistan

    Falak Sher

Authors
  1. Joost-Pieter Katoen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Falak Sher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Falak Sher .

Editor information

Editors and Affiliations

  1. Reykjavik University, Reykjavik, Iceland

    Luca Aceto

  2. Aalborg University, Aalborg, Denmark

    Giorgio Bacci

  3. Aalborg University, Aalborg, Denmark

    Giovanni Bacci

  4. Reykjavík University, Reykjavik, Iceland

    Anna Ingólfsdóttir

  5. Inria Campus Beaulieu, Rennes, France

    Axel Legay

  6. Aalborg University, Aalborg, Denmark

    Radu Mardare

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Katoen, JP., Sher, F. (2017). Modal Stochastic Games. In: Aceto, L., Bacci, G., Bacci, G., Ingólfsdóttir, A., Legay, A., Mardare, R. (eds) Models, Algorithms, Logics and Tools. Lecture Notes in Computer Science(), vol 10460. Springer, Cham. https://doi.org/10.1007/978-3-319-63121-9_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-63121-9_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-63120-2

  • Online ISBN: 978-3-319-63121-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation