Skip to main content
Springer Nature Link
Log in

Scenario Approach for Parametric Markov Models

  • Conference paper
  • First Online:
Automated Technology for Verification and Analysis (ATVA 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14215))

Abstract

In this paper, we propose an approximating framework for analyzing parametric Markov models. Instead of computing complex rational functions encoding the reachability probability and the reward values of the parametric model, we exploit the scenario approach to synthesize a relatively simple polynomial approximation. The approximation is probably approximately correct (PAC), meaning that with high confidence, the approximating function is close to the actual function with an allowable error. With the PAC approximations, one can check properties of the parametric Markov models. We show that the scenario approach can also be used to check PRCTL properties directly – without synthesizing the polynomial at first hand. We have implemented our algorithm in a prototype tool and conducted thorough experiments. The experimental results demonstrate that our tool is able to compute polynomials for more benchmarks than state-of-the-art tools such as PRISM and Storm, confirming the efficacy of our PAC-based synthesis.

A. Turrini—Co-first author.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Data Availability Statement

An environment with the tools and data used for the experimental evaluation presented in this work is available in the following Zenodo repository: https://doi.org/10.5281/zenodo.8181117.

Notes

  1. 1.

    https://github.com/iscas-tis/PacPMA/.

  2. 2.

    https://github.com/moves-rwth/storm/.

References

  1. Andova, S., Hermanns, H., Katoen, J.-P.: Discrete-time rewards model-checked. In: Larsen, K.G., Niebert, P. (eds.) FORMATS 2003. LNCS, vol. 2791, pp. 88–104. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-40903-8_8

    Chapter  Google Scholar 

  2. Badings, T.S., Cubuktepe, M., Jansen, N., Junges, S., Katoen, J., Topcu, U.: Scenario-based verification of uncertain parametric MDPs. Int. J. Softw. Tools Technol. Transf. 24(5), 803–819 (2022). https://doi.org/10.1007/s10009-022-00673-z

    Article  MATH  Google Scholar 

  3. Badings, T.S., Jansen, N., Junges, S., Stoelinga, M., Volk, M.: Sampling-based verification of CTMCs with uncertain rates. In: Shoham, S., Vizel, Y. (eds.) CAV 2022 Part II. LNCS, vol. 13372, pp. 26–47. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-13188-2_2

    Chapter  MATH  Google Scholar 

  4. Bai, H., Cai, S., Ye, N., Hsu, D., Lee, W.S.: Intention-aware online POMDP planning for autonomous driving in a crowd. In: IEEE International Conference on Robotics and Automation, ICRA 2015, Seattle, WA, USA, 26–30 May, 2015, pp. 454–460. IEEE (2015). https://doi.org/10.1109/ICRA.2015.7139219

  5. Baier, C., Hensel, C., Hutschenreiter, L., Junges, S., Katoen, J., Klein, J.: Parametric Markov chains: PCTL complexity and fraction-free Gaussian elimination. Inf. Comput. 272, 104504 (2020). https://doi.org/10.1016/j.ic.2019.104504

    Article  MathSciNet  MATH  Google Scholar 

  6. Baier, C., Katoen, J.: Principles of Model Checking. MIT Press, Cambridge (2008)

    MATH  Google Scholar 

  7. Ben-Tal, A., Nemirovski, A.: Robust convex optimization. Math. Oper. Res. 23(4), 769–805 (1998). https://doi.org/10.1287/moor.23.4.769

    Article  MathSciNet  MATH  Google Scholar 

  8. Ben-Tal, A., Nemirovski, A.: Robust solutions of uncertain linear programs. Oper. Res. Lett. 25(1), 1–13 (1999). https://doi.org/10.1016/S0167-6377(99)00016-4

    Article  MathSciNet  MATH  Google Scholar 

  9. Beyer, D., Löwe, S., Wendler, P.: Reliable benchmarking: requirements and solutions. Int. J. Softw. Tools Technol. Transfer 21(1), 1–29 (2017). https://doi.org/10.1007/s10009-017-0469-y

    Article  Google Scholar 

  10. Bianco, A., de Alfaro, L.: Model checking of probabilistic and nondeterministic systems. In: Thiagarajan, P.S. (ed.) FSTTCS 1995. LNCS, vol. 1026, pp. 499–513. Springer, Heidelberg (1995). https://doi.org/10.1007/3-540-60692-0_70

    Chapter  MATH  Google Scholar 

  11. Blackwell, D.: On the functional equation of dynamic programming. J. Math. Anal. Appl. 2(2), 273–276 (1961)

    Article  MathSciNet  MATH  Google Scholar 

  12. Boggess, A., Narcowich, F.J.: A First Course in Wavelets with Fourier Analysis. Wiley, Hoboken (2015)

    MATH  Google Scholar 

  13. Bortolussi, L., Milios, D., Sanguinetti, G.: Smoothed model checking for uncertain continuous-time Markov chains. Inf. Comput. 247, 235–253 (2016). https://doi.org/10.1016/j.ic.2016.01.004

    Article  MathSciNet  MATH  Google Scholar 

  14. Bortolussi, L., Silvetti, S.: Bayesian statistical parameter synthesis for linear temporal properties of stochastic models. In: Beyer, D., Huisman, M. (eds.) TACAS 2018. LNCS, vol. 10806, pp. 396–413. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-89963-3_23

    Chapter  MATH  Google Scholar 

  15. Calafiore, G.C., Campi, M.C.: Uncertain convex programs: randomized solutions and confidence levels. Math. Program. 102(1), 25–46 (2005). https://doi.org/10.1007/s10107-003-0499-y

    Article  MathSciNet  MATH  Google Scholar 

  16. Calafiore, G.C., Campi, M.C.: The scenario approach to robust control design. IEEE Trans. Autom. Control. 51(5), 742–753 (2006). https://doi.org/10.1109/TAC.2006.875041

    Article  MathSciNet  MATH  Google Scholar 

  17. Campi, M.C., Garatti, S.: A sampling-and-discarding approach to chance-constrained optimization: feasibility and optimality. J. Optim. Theory Appl. 148(2), 257–280 (2011). https://doi.org/10.1007/s10957-010-9754-6

    Article  MathSciNet  MATH  Google Scholar 

  18. Campi, M.C., Garatti, S., Prandini, M.: The scenario approach for systems and control design. Annu. Rev. Control. 33(2), 149–157 (2009). https://doi.org/10.1016/j.arcontrol.2009.07.001

    Article  Google Scholar 

  19. Carè, A., Garatti, S., Campi, M.C.: Scenario min-max optimization and the risk of empirical costs. SIAM J. Optim. 25(4), 2061–2080 (2015). https://doi.org/10.1137/130928546

    Article  MathSciNet  MATH  Google Scholar 

  20. Clarke, E.M., Henzinger, T.A., Veith, H., Bloem, R. (eds.): Handbook of Model Checking. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-10575-8

  21. Conway, J.B.: A Course in Functional Analysis, vol. 96. Springer, Cham (2019)

    Google Scholar 

  22. Daws, C.: Symbolic and parametric model checking of discrete-time Markov chains. In: Liu, Z., Araki, K. (eds.) ICTAC 2004. LNCS, vol. 3407, pp. 280–294. Springer, Heidelberg (2005). https://doi.org/10.1007/978-3-540-31862-0_21

    Chapter  MATH  Google Scholar 

  23. Dehnert, C., et al.: PROPhESY: A PRObabilistic ParamEter SYnthesis Tool. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 214–231. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21690-4_13

    Chapter  Google Scholar 

  24. Dehnert, C., Junges, S., Katoen, J., Volk, M.: A storm is coming: A modern probabilistic model checker. In: Majumdar, R., Kuncak, V. (eds.) Computer Aided Verification. CAV 2017. LNCS vol. 10427, pp. 592–600. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63390-9_31

  25. Dubins, L.E., Savage, L.: How to Gamble If You Must. McGraw-Hill (1965)

    Google Scholar 

  26. Fang, X., Calinescu, R., Gerasimou, S., Alhwikem, F.: Fast parametric model checking through model fragmentation. In: 43rd IEEE/ACM International Conference on Software Engineering, ICSE 2021, Madrid, Spain, 22–30 May 2021, pp. 835–846. IEEE (2021). https://doi.org/10.1109/ICSE43902.2021.00081

  27. Gainer, P., Hahn, E.M., Schewe, S.: Accelerated model checking of parametric Markov chains. In: Lahiri, S.K., Wang, C. (eds.) ATVA 2018. LNCS, vol. 11138, pp. 300–316. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01090-4_18

    Chapter  Google Scholar 

  28. Garavel, H., Lang, F., Mateescu, R., Serwe, W.: CADP 2011: a toolbox for the construction and analysis of distributed processes. Int. J. Softw. Tools Technol. Transf. 15(2), 89–107 (2013). https://doi.org/10.1007/s10009-012-0244-z

    Article  MATH  Google Scholar 

  29. Ghaoui, L.E., Oustry, F., Lebret, H.: Robust solutions to uncertain semidefinite programs. SIAM J. Optim. 9(1), 33–52 (1998). https://doi.org/10.1137/S1052623496305717

    Article  MathSciNet  MATH  Google Scholar 

  30. Hahn, E.M., Han, T., Zhang, L.: Synthesis for PCTL in parametric Markov decision processes. In: Bobaru, M., Havelund, K., Holzmann, G.J., Joshi, R. (eds.) NFM 2011. LNCS, vol. 6617, pp. 146–161. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20398-5_12

    Chapter  Google Scholar 

  31. Hahn, E.M., Hermanns, H., Wachter, B., Zhang, L.: PARAM: a model checker for parametric Markov models. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 660–664. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14295-6_56

    Chapter  Google Scholar 

  32. Hahn, E.M., Hermanns, H., Zhang, L.: Probabilistic reachability for parametric Markov models. Int. J. Softw. Tools Technol. Transf. 13(1), 3–19 (2011). https://doi.org/10.1007/s10009-010-0146-x

    Article  Google Scholar 

  33. Hahn, E.M., Li, Y., Schewe, S., Turrini, A., Zhang, L.: iscasMc: a web-based probabilistic model checker. In: Jones, C., Pihlajasaari, P., Sun, J. (eds.) FM 2014. LNCS, vol. 8442, pp. 312–317. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-06410-9_22

    Chapter  Google Scholar 

  34. Han, T.: Diagnosis, synthesis and analysis of probabilistic models. In: Bernstein, A., et al. (eds.) Ausgezeichnete Informatikdissertationen 2009, LNI, vol. D-10, pp. 81–90. GI (2009). https://dl.gi.de/handle/20.500.12116/33657

  35. Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Formal Aspects Comput. 6(5), 512–535 (1994). https://doi.org/10.1007/BF01211866

    Article  MATH  Google Scholar 

  36. Hensel, C., Junges, S., Katoen, J., Quatmann, T., Volk, M.: The probabilistic model checker storm. Int. J. Softw. Tools Technol. Transf. 24(4), 589–610 (2022). https://doi.org/10.1007/s10009-021-00633-z

    Article  Google Scholar 

  37. von Hilgers, P., Langville, A.N.: The five greatest applications of Markov chains. In: Proceedings of the Markov Anniversary Meeting, pp. 155–168 (2006)

    Google Scholar 

  38. Hopcroft, J.E., Motwani, R., Ullman, J.D.: Introduction to Automata Theory, Languages, and Computation, 3rd edn. Addison-Wesley, Pearson international edition (2007)

    Google Scholar 

  39. Jansen, N., et al.: Accelerating parametric probabilistic verification. In: Norman, G., Sanders, W. (eds.) QEST 2014. LNCS, vol. 8657, pp. 404–420. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10696-0_31

    Chapter  Google Scholar 

  40. Jansen, N., Junges, S., Katoen, J.: Parameter synthesis in Markov models: a gentle survey. In: Raskin, J., Chatterjee, K., Doyen, L., Majumdar, R. (eds.) Principles of Systems Design. LNCS, vol. 13660, pp. 407–437. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-22337-2_20

    Chapter  Google Scholar 

  41. Junges, S., Katoen, J., Pérez, G.A., Winkler, T.: The complexity of reachability in parametric Markov decision processes. J. Comput. Syst. Sci. 119, 183–210 (2021). https://doi.org/10.1016/j.jcss.2021.02.006

    Article  MathSciNet  MATH  Google Scholar 

  42. Katoen, J., Zapreev, I.S., Hahn, E.M., Hermanns, H., Jansen, D.N.: The ins and outs of the probabilistic model checker MRMC. Perform. Evaluation 68(2), 90–104 (2011). https://doi.org/10.1016/j.peva.2010.04.001

    Article  Google Scholar 

  43. Kwiatkowska, M., Norman, G., Parker, D.: Stochastic model checking. In: Bernardo, M., Hillston, J. (eds.) SFM 2007. LNCS, vol. 4486, pp. 220–270. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-72522-0_6

    Chapter  Google Scholar 

  44. Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22110-1_47

    Chapter  Google Scholar 

  45. Kwiatkowska, M.Z., Norman, G., Parker, D.: The PRISM benchmark suite. In: Ninth International Conference on Quantitative Evaluation of Systems, QEST 2012, London, United Kingdom, September 17–20, 2012, pp. 203–204. IEEE Computer Society (2012). https://doi.org/10.1109/QEST.2012.14

  46. Lasserre, J.B.: A semidefinite programming approach to the generalized problem of moments. Math. Program. 112(1), 65–92 (2008). https://doi.org/10.1007/s10107-006-0085-1

    Article  MathSciNet  MATH  Google Scholar 

  47. Lasserre, J.B.: Moments, Positive Polynomials and their Applications, vol. 1. World Scientific, Singapore (2009)

    Google Scholar 

  48. Li, R., Yang, P., Huang, C., Sun, Y., Xue, B., Zhang, L.: Towards practical robustness analysis for DNNs based on PAC-model learning. In: 44th IEEE/ACM 44th International Conference on Software Engineering, ICSE 2022, Pittsburgh, PA, USA, May 25–27, 2022, pp. 2189–2201. ACM (2022). https://doi.org/10.1145/3510003.3510143

  49. Liu, Y., Turrini, A., Hahn, E.M., Xue, B., Zhang, L.: Scenario approach for parametric Markov models. CoRR abs/2304.08330 (2023). https://doi.org/10.48550/arXiv.2304.08330

  50. Margellos, K., Goulart, P., Lygeros, J.: On the road between robust optimization and the scenario approach for chance constrained optimization problems. IEEE Trans. Autom. Control. 59(8), 2258–2263 (2014). https://doi.org/10.1109/TAC.2014.2303232

    Article  MathSciNet  MATH  Google Scholar 

  51. Pardalos, P.M., Ye, Y., Han, C.G.: Algorithms for the solution of quadratic knapsack problems. Linear Algebra Appl. 152, 69–91 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  52. Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley Series in Probability and Statistics, Wiley, Hoboken (1994). https://doi.org/10.1002/9780470316887

  53. Quatmann, T., Dehnert, C., Jansen, N., Junges, S., Katoen, J.-P.: Parameter synthesis for markov models: faster than ever. In: Artho, C., Legay, A., Peled, D. (eds.) ATVA 2016. LNCS, vol. 9938, pp. 50–67. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46520-3_4

    Chapter  MATH  Google Scholar 

  54. Rockafellar, R.T.: Convex Analysis, vol. 11. Princeton University Press, Princeton (1997)

    MATH  Google Scholar 

  55. Rudin, W.: Principles of Mathematical Analysis, vol. 3. McGraw-Hill, New York (1976)

    MATH  Google Scholar 

  56. Sahni, S.: Computationally related problems. SIAM J. Comput. 3(4), 262–279 (1974). https://doi.org/10.1137/0203021

    Article  MathSciNet  MATH  Google Scholar 

  57. Spel, J., Junges, S., Katoen, J.-P.: Finding provably optimal Markov chains. In: TACAS 2021. LNCS, vol. 12651, pp. 173–190. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72016-2_10

    Chapter  Google Scholar 

  58. Vayanos, P., Kuhn, D., Rustem, B.: A constraint sampling approach for multi-stage robust optimization. Automatica 48(3), 459–471 (2012). https://doi.org/10.1016/j.automatica.2011.12.002

    Article  MathSciNet  MATH  Google Scholar 

  59. Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8, 279–292 (1992). https://doi.org/10.1007/BF00992698

    Article  MATH  Google Scholar 

  60. Xue, B., Zhang, M., Easwaran, A., Li, Q.: PAC model checking of black-box continuous-time dynamical systems. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 39(11), 3944–3955 (2020). https://doi.org/10.1109/TCAD.2020.3012251

  61. Yang, J., Ye, K., Zhi, L.: Computing sparse Fourier sum of squares on finite Abelian groups in quasi-linear time. CoRR abs/2201.03912 (2022)

    Google Scholar 

Download references

Acknowledgements

We thank the anonymous reviewers for their useful remarks that helped us improve the quality of the paper. Work supported in part by the CAS Project for Young Scientists in Basic Research under grant No. YSBR-040, NSFC under grant No. 61836005, the CAS Pioneer Hundred Talents Program, the ISCAS New Cultivation Project ISCAS-PYFX-202201, and the ERC Consolidator Grant 864075 (CAESAR).

This project is part of the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant no. 101008233.

Author information

Authors and Affiliations

  1. State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, China

    Ying Liu, Andrea Turrini, Bai Xue & Lijun Zhang

  2. University of Chinese Academy of Sciences, Beijing, China

    Ying Liu & Lijun Zhang

  3. Institute of Intelligent Software Guangzhou, Guangzhou, China

    Andrea Turrini & Lijun Zhang

  4. Formal Methods and Tools, University of Twente, Enschede, The Netherlands

    Ernst Moritz Hahn

Authors
  1. Ying Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea Turrini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ernst Moritz Hahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bai Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lijun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Bai Xue or Lijun Zhang .

Editor information

Editors and Affiliations

  1. Université Sorbonne Paris Nord, Villetaneuse, France

    Étienne André

  2. Singapore Management University, Singapore, Singapore

    Jun Sun

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, Y., Turrini, A., Hahn, E.M., Xue, B., Zhang, L. (2023). Scenario Approach for Parametric Markov Models. In: André, É., Sun, J. (eds) Automated Technology for Verification and Analysis. ATVA 2023. Lecture Notes in Computer Science, vol 14215. Springer, Cham. https://doi.org/10.1007/978-3-031-45329-8_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-45329-8_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-45328-1

  • Online ISBN: 978-3-031-45329-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation