Skip to main content

Adaptive Stopping Algorithms Based on Concentration Inequalities

  • Conference paper
  • First Online:
Bridging the Gap Between AI and Reality (AISoLA 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15217))

Included in the following conference series:

  • 75 Accesses

Abstract

Sampling is a key step of stochastic methods. In some contexts, minimizing the sample size can be of critical importance and determine by itself the viability of various approaches. Rethinking the way samples are generated can even lead to new algorithms, better suited than their alternatives to tackle specific real-world problems for which the sampling task is a costly step of the estimation process. For instance, strategies of formal verification and statistical model checking can be greatly improved by the use of adaptive stopping algorithms. Instead of initially computing the necessary sample size, those algorithms generate the samples progressively while continuously monitoring their progress along the way, allowing them to stop the sampling process as soon as possible. We present a generalization of two existing adaptive stopping algorithms for statistical model checking, and we show how this generalization can be exploited to derive tailor-made variations for specific use cases.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Agha, G., Palmskog, K.: A survey of statistical model checking. ACM Trans. Model. Comput. Simulat. 28(1), 1–39 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  2. Audibert, J.Y., Munos, R., Szepesvári, C.: Variance estimates and exploration function in multi-armed bandit. In: CERTIS Research Report 07–31. Citeseer (2007)

    Google Scholar 

  3. Baier, C., Katoen, J.P.: Principles of Model Checking. MIT Press (2008)

    Google Scholar 

  4. Basile, D., ter Beek, M.H., Ferrari, A., Legay, A.: Exploring the ERTMS/ETCS full moving block specification: an experience with formal methods. Int. J. Softw. Tools Technol. Transf. 24(3), 351–370 (2022)

    Google Scholar 

  5. Behrmann, G., David, A., Larsen, K.G.: A tutorial on uppaal. In: Formal Methods for the Design of Real-time Systems, pp. 200–236 (2004)

    Google Scholar 

  6. Boucheron, S., Lugosi, G., Bousquet, O.: Concentration inequalities. In: Summer School on Machine Learning, pp. 208–240. Springer, Heidelberg (2003)

    Google Scholar 

  7. Bowman, H., Faconti, G., Katoen, J.P., Latella, D., Massink, M.: Automatic verification of a lip-synchronisation protocol using uppaal. Formal Aspects Comput. 10, 550–575 (1998)

    Article  MATH  Google Scholar 

  8. Broy, M., Jonsson, B., Katoen, J.P., Leucker, M., Pretschner, A.: Model-Based Testing of Reactive Systems. LNCS, vol. 3472. Springer, Heidelberg (2005)

    Google Scholar 

  9. Busa-Fekete, R., Szorenyi, B., Cheng, W., Weng, P., Hüllermeier, E.: Top-k selection based on adaptive sampling of noisy preferences. In: International Conference on Machine Learning, pp. 1094–1102. PMLR (2013)

    Google Scholar 

  10. Casella, G., Berger, R.L.: Statistical inference. In: Cengage Learning (2021)

    Google Scholar 

  11. Chung Graham, F., Lu, L.: Complex Graphs and Networks. American Mathematical Society (2006)

    Google Scholar 

  12. Clarke, E.M., Klieber, W., Nováček, M., Zuliani, P.: Model checking and the state explosion problem. In: LASER Summer School on Software Engineering, pp. 1–30. Springer, Heidelberg (2011)

    Google Scholar 

  13. Clarke, E.M., Zuliani, P.: Statistical model checking for cyber-physical systems. In: International Symposium on Automated Technology for Verification and Analysis, pp. 1–12. Springer, Heidelberg (2011)

    Google Scholar 

  14. Dagum, P., Karp, R., Luby, M., Ross, S.: An optimal algorithm for Monte Carlo estimation. SIAM J. Comput. 29(5), 1484–1496 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  15. Ding, X., Smith, S.L., Belta, C., Rus, D.: Optimal control of Markov decision processes with linear temporal logic constraints. IEEE Trans. Autom. Control 59(5), 1244–1257 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  16. Domingo, C., Gavalda, R., Watanabe, O.: Adaptive sampling methods for scaling up knowledge discovery algorithms. Data Min. Knowl. Disc. 6, 131–152 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  17. D’Argenio, P., Legay, A., Sedwards, S., Traonouez, L.-M.: Smart sampling for lightweight verification of Markov decision processes. Int. J. Softw. Tools Technol. Transfer 17(4), 469–484 (2015). https://doi.org/10.1007/s10009-015-0383-0

    Article  MATH  Google Scholar 

  18. Heidrich-Meisner, V., Igel, C.: Hoeffding and Bernstein races for selecting policies in evolutionary direct policy search. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 401–408 (2009)

    Google Scholar 

  19. Heidrich-Meisner, V., Igel, C.: Neuroevolution strategies for episodic reinforcement learning. J. Algorithms 64(4), 152–168 (2009)

    Article  MATH  Google Scholar 

  20. Henriques, D., Martins, J.G., Zuliani, P., Platzer, A., Clarke, E.M.: Statistical model checking for Markov decision processes. In: 2012 Ninth International Conference on Quantitative Evaluation of Systems, pp. 84–93. IEEE (2012)

    Google Scholar 

  21. Hérault, T., Lassaigne, R., Magniette, F., Peyronnet, S.: Approximate probabilistic model checking. In: Steffen, B., Levi, G. (eds.) VMCAI 2004. LNCS, vol. 2937, pp. 73–84. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24622-0_8

    Chapter  MATH  Google Scholar 

  22. Jegourel, C., Legay, A., Sedwards, S.: Importance splitting for statistical model checking rare properties. In: Sharygina, N., Veith, H. (eds.) CAV 2013. LNCS, vol. 8044, pp. 576–591. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39799-8_38

    Chapter  MATH  Google Scholar 

  23. Kahn, H.: Random Sampling (Monte Carlo) Techniques in Neutron Attenuation Problems. I. Nucleonics (US) Ceased Publication 6 (See also NSA 3-990) (1950)

    Google Scholar 

  24. Kahn, H., Harris, T.E.: Estimation of particle transmission by random sampling. Natl. Bureau Stand. Appl. Math. Ser. 12, 27–30 (1951)

    MATH  Google Scholar 

  25. Kahn, H., Marshall, A.W.: Methods of reducing sample size in Monte Carlo computations. J. Oper. Res. Soc. Am. 1(5), 263–278 (1953)

    MATH  Google Scholar 

  26. Kwiatkowska, M., Norman, G., Parker, D.: Prism: probabilistic model checking for performance and reliability analysis. ACM SIGMETRICS Perform. Eval. Rev. 36(4), 40–45 (2009)

    Article  MATH  Google Scholar 

  27. Kwiatkowska, M., Norman, G., Sproston, J., Wang, F.: Symbolic model checking for probabilistic timed automata. Inf. Comput. 205(7), 1027–1077 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  28. Larsen, K.G., Legay, A.: Statistical model checking: past, present, and future. In: Margaria, T., Steffen, B. (eds.) ISoLA 2016. LNCS, vol. 9952, pp. 3–15. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-47166-2_1

    Chapter  MATH  Google Scholar 

  29. Legay, A., Lukina, A., Traonouez, L.M., Yang, J., Smolka, S.A., Grosu, R.: Statistical model checking. In: Computing and Software Science: State of the Art and Perspectives, pp. 478–504. Springer, Cham (2019)

    Google Scholar 

  30. Legay, A., Sedwards, S., Traonouez, L.M.: Plasma lab: a modular statistical model checking platform. In: International Symposium on Leveraging Applications of Formal Methods. pp. 77–93. Springer (2016)

    Google Scholar 

  31. Lindahl, M., Pettersson, P., Yi, W.: Formal design and analysis of a gear controller. In: Steffen, B. (ed.) TACAS 1998. LNCS, vol. 1384, pp. 281–297. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0054178

    Chapter  MATH  Google Scholar 

  32. Mnih, V., Szepesvári, C., Audibert, J.Y.: Empirical Bernstein stopping. In: Proceedings of the 25th International Conference on Machine Learning, pp. 672–679 (2008)

    Google Scholar 

  33. Okamoto, M.: Some inequalities relating to the partial sum of binomial probabilities. Ann. Inst. Stat. Math. 10, 29–35 (1959)

    Article  MathSciNet  MATH  Google Scholar 

  34. Pappagallo, A., Massini, A., Tronci, E.: Monte Carlo based statistical model checking of cyber-physical systems: a review. Information 11(12), 588 (2020)

    Article  MATH  Google Scholar 

  35. Parmentier, M., Legay, A., Chenoy, F.: Optimized smart sampling. In: International Conference on Bridging the Gap between AI and Reality, pp. 171–187. Springer, Cham (2023)

    Google Scholar 

  36. Rosenbluth, M.N., Rosenbluth, A.W.: Monte Carlo calculation of the average extension of molecular chains. J. Chem. Phys. 23(2), 356–359 (1955)

    Article  MATH  Google Scholar 

  37. Sen, K., Viswanathan, M., Agha, G.: On statistical model checking of stochastic systems. In: Etessami, K., Rajamani, S.K. (eds.) CAV 2005. LNCS, vol. 3576, pp. 266–280. Springer, Heidelberg (2005). https://doi.org/10.1007/11513988_26

    Chapter  Google Scholar 

  38. Wasserman, L.: All of Statistics: A Concise Course in Statistical Inference. Springer, New York (2004)

    Google Scholar 

  39. Zuliani, P., Platzer, A., Clarke, E.M.: Bayesian statistical model checking with application to Simulink/Stateflow verification. In: Proceedings of the 13th ACM International Conference on Hybrid Systems: Computation and Control, pp. 243–252 (2010)

    Google Scholar 

Download references

Acknowledgements

M. Parmentier is funded by a FNRS PhD Grant and by the UCLouvain.

A. Legay is funded by a FNRS PDR - T013721.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Axel Legay .

Editor information

Editors and Affiliations

Appendix: Concentration Inequalities

Appendix: Concentration Inequalities

This is a nonexhaustive list of some of the most well-known and useful concentration inequalities. Most of them are regarded as relatively elementary results in statistics [6, 10, 38].

Let X be a random variable.

  • Markov’s inequality (X must be almost surely nonnegative):

    $$\forall \epsilon > 0:P(X \ge \epsilon ) \le \frac{E(X)}{\epsilon }$$
  • Chebyshev’s inequality (E(X) and V(X) must be finite):

    $$\forall \epsilon > 0:P(|X-E(X)| \ge \epsilon ) \le \frac{V(X)}{{\epsilon }^2}$$
  • Chernoff bound (the moment generating function of X, \(M_X(t) = E(e^{tX})\), must be well-defined and finite):

    $$\forall \epsilon > 0:P(X \ge \epsilon ) \le \frac{E(e^{tX})}{e^{t\epsilon }}$$

Now let \(X = S_n = X_1 + ... + X_n\) be the sum of n independent random variables, with each \(X_i\) almost surely bounded within \([a_i,b_i]\). Let \(R_i = b_i - a_i\) be the range of \(X_i\). Let be \(R = \underset{1\le i\le n}{\max }(R_i)\).

  • Hoeffding’s inequality:

    $$\forall \epsilon > 0:P(|S_n - E(S_n)| \ge \epsilon ) \le 2\exp \left( -\frac{2{\epsilon }^2}{\sum _{i=1}^{n} (b_i - a_i)^2}\right) $$
  • Bernstein’s inequality:

    $$\forall \epsilon > 0:{P(|S_n - E(S_n)| \ge \epsilon ) \le 2\exp \left( -\frac{{\epsilon }^2}{2V(S_n)+(2/3)\epsilon R}\right) }$$
  • Both Azuma’s Inequality and McDiamid’s inequality are equivalent to Hoeffding’s inequality with those assumptions:

    $$\forall \epsilon > 0:P(|S_n - E(S_n)| \ge \epsilon ) \le 2\exp \left( -\frac{2{\epsilon }^2}{\sum _{i=1}^{n} (b_i - a_i)^2}\right) $$
  • Benett’s inequality (with \(f(x) = (1+x)\log (1+x)-x\)):

    $$\forall \epsilon > 0:{P(|S_n - E(S_n)| \ge \epsilon ) \le 2\exp \left( -\frac{V(S_n)}{R^2} f\left( \frac{\epsilon R}{V(S_n)}\right) \right) }$$

Additionally, if the random variables \(X_1\), ..., \(X_n\) are independent and identically distributed (i.i.d.), the central limit theorem itself can be interpreted as a concentration inequality: as n grows, \(S_n - E(S_n)\) can be approximated with \(Z \sim \mathcal {N}(0,V(S_n))\), which implies:

$$\forall \epsilon > 0:P(|S_n - E(S_n)| \ge \epsilon ) \simeq P(|Z| \ge \epsilon ) = 2\int \limits _{\epsilon }^{\infty } \frac{1}{\sqrt{2\pi V(S_n)}}\exp \left( -\frac{x^2}{2V(S_n)}\right) dx$$

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Parmentier, M., Legay, A. (2025). Adaptive Stopping Algorithms Based on Concentration Inequalities. In: Steffen, B. (eds) Bridging the Gap Between AI and Reality. AISoLA 2024. Lecture Notes in Computer Science, vol 15217. Springer, Cham. https://doi.org/10.1007/978-3-031-75434-0_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-75434-0_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-75433-3

  • Online ISBN: 978-3-031-75434-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics