Skip to main content
SpringerLink
Log in

Practical policy iterations

A practical use of policy iterations for static analysis: the quadratic case

  • Published:
Formal Methods in System Design Aims and scope Submit manuscript

Abstract

Policy iterations is a technique based on game theory that relies on a sequence of numerical optimization queries to compute the fixpoint of a set of equations. It has been proposed to support the static analysis of programs as an alternative to widening, when the latter is ineffective. This happens for instance with highly numerical codes, such as found at cores of control command applications. In this paper we present a complete, yet practical, description of the use of policy iteration in this context. We recall the rationale behind policy iteration and address required steps towards an automatic use of it: synthesis of numerical templates, floating point semantics of the analyzed program and issues with the accuracy of numerical solvers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. The term strategy is also used in the literature, with equivalent meaning.

  2. Although the \(k\)-inductive invariants can be made (1)-inductive by adding extra variables, representing past values of program variables, in their expression.

  3. Like the one in Fig. 1.

  4. All figures are rounded to the fourth digit.

  5. Although the minimum volume (Löwner-Johns) ellipsoid [6], Section 8.4] could be a reasonable choice.

  6. \(\vee \) is often used instead in the policy iteration literature.

  7. Or a large enough guess can be used. Thanks to the fast convergence of min-policy iterations, there is often no need for this postfixpoint to be close from the fixpoint eventually computed.

  8. More precisely, first determine which \(b_{i,j}\) are \(\pm \infty \) in the least fixpoint in \({\overline{\mathbb {R}}}^{np}\) greater than \(b_i\), then compute a greatest fixpoint for the remaining values in \({\mathbb {R}}\).

  9. More precisely, for a given policy \(\overline{F}_{i+1}\), once determined which \(b_{i,j}\) are \(\pm \infty \) there is a unique greatest fixpoint for the remaining \(b_{i,j} \in {\mathbb {R}}\), hence finitely many possible \(b_{i+1}\).

  10. There is usually no best ellipsoidal invariant, so we have to resort on a heuristic.

  11. Moreover, \(x^2\) being an homogeneous degree two polynomial is easier to express in semi-definite programs than linear constraints which would require an extra dimension to encode linear terms.

  12. Although they are clearly the maximal values of each template under constraint \(0 \le x_1 \le 1 \wedge 0 \le x_2 \le 1\).

  13. For, denoting \(p\) the previous degree two polynomial, \(\displaystyle \lim _{x \rightarrow \infty } p(x) = -\infty \), whatever the values of \(\lambda _1\) and \(\lambda _2\).

  14. Only one constraint here for ease of exposition. Everything works the same with multiple constraints.

  15. Thanks to Timothy Wang for pointing this to us.

  16. Although, in our case, this positive definiteness check only accounts for a very small part of the total analysis time. Thus, the eventual overhead would remain limited.

  17. Usual implementation of type double in C.

  18. Order of evaluation matters since floating point addition is not associative.

  19. A similar proof can be performed if the sum is not computed in this left-right order.

  20. The relative difference between the \(b_{v',j}\) and the \(b'_{v',j}\) or the \(c\) and \(c'\) never exceeded \(10^{-10}\) in our experiments (to be compared to the \(10^{-4}\) padding previously applied).

  21. This is explained by the fact that max-policies have to solve larger SDP problems, incurring more numerical difficulties [20], Conclusion] (c.f., Remarks 4, page 14 and 6, page 16).

References

  1. Adjé A, Gaubert S, Goubault E (2010) Coupling policy iteration with semi-definite relaxation to compute accurate numerical invariants in static analysis. In: ESOP, pp 23–42

  2. Alegre F, Féron E, Pande S (2009) Using ellipsoidal domains to analyze control systems software. arXiv:0909.1977

  3. Boldo S, Melquiond G (2011) Flocq: a unified library for proving floating-point algorithms in Coq. In: Proceedings of the 20th IEEE symposium on computer arithmetic. Tübingen, pp 243–252

  4. Bouissou O, Seladji Y, Chapoutot A (2012) Acceleration of the abstract fixpoint computation in numerical program analysis. J Symb Comput 47(12):1479–1511

    Article  MATH  MathSciNet  Google Scholar 

  5. Boyd S, El Ghaoui L, Féron E, Balakrishnan V (1994) Linear matrix inequalities in system and control theory, volume 15 of SIAM. SIAM, Philadelphia

    Book  Google Scholar 

  6. Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  7. Champion A, Delmas R, Dierkes M, Garoche P-L, Jobredeaux R, Roux P (2013) Formal methods for the analysis of critical control systems models: combining non-linear and linear analyses. In: Charles P, Michael D, (eds), Formal methods for industrial critical systems—18th international workshop, FMICS 2013, Madrid, Spain, September 23–24, 2013. Proceedings, volume 8187 of Lecture Notes in Computer Science, pp 1–16. Springer

  8. Costan A, Gaubert S, Goubault E, Martel M, Putot S (2005) A policy iteration algorithm for computing fixed points in static analysis of programs. In: CAV, pp 462–475

  9. Cousot P, Cousot R (1977) Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: POPL, pp 238–252

  10. Cousot P, Cousot R (1979) Systematic design of program analysis frameworks. In: POPL, pp 269–282

  11. Cousot P, Cousot R (1992) Abstract interpretation frameworks. J Log Comput 2(4):511–547

    Article  MATH  MathSciNet  Google Scholar 

  12. Cousot P, Halbwachs N (1978) Automatic discovery of linear restraints among variables of a program. In: POPL, pp 84–96

  13. Feautrier P, Gonnord L (2010) Accelerated invariant generation for c programs with aspic and c2fsm. Electron Notes Theor Comput Sci 267(2):3–13

    Article  Google Scholar 

  14. Feret J (2004) Static analysis of digital filters. In: ESOP, number 2986 in LNCS. Springer

  15. Feret J (2005) Numerical abstract domains for digital filters. In: International workshop on Numerical and Symbolic Abstract Domains (NSAD)

  16. Féron E (2010) From control systems to control software. IEEE Control Syst 30(6):50–71

    Article  MathSciNet  Google Scholar 

  17. Gaubert S, Goubault E, Taly A, Zennou S (2007) Static analysis by policy iteration on relational domains. In: ESOP, pp 237–252

  18. Gawlitza T, Seidl H (2007) Precise fixpoint computation through strategy iteration. In: ESOP, pp 300–315

  19. Gawlitza TM, Seidl H (2010) Computing relaxed abstract semantics w.r.t. quadratic zones precisely. In: SAS, pp 271–286

  20. Gawlitza TM, Seidl H, Adjé A, Gaubert S, Goubault E (2012) Abstract interpretation meets convex optimization. J Symb Comput 47(12):1416–1446

    Article  MATH  Google Scholar 

  21. Ghorbal K, Goubault E, Putot S (2009) The zonotope abstract domain taylor1+. In: CAV, pp 627–633

  22. Gopan D, Reps TW (2006) Lookahead widening. In: CAV, pp 452–466

  23. Goubault E, Putot S (2011) Static analysis of finite precision computations. In: VMCAI, pp 232–247

  24. Haddad WM, Chellaboina VS (2008) Nonlinear dynamical systems and control: a lyapunov-based approach. Princeton University Press, Princeton

    Google Scholar 

  25. Halbwachs N, Henry J (2012) When the decreasing sequence fails. In: SAS, pp 198–213

  26. Halbwachs Nicolas, Proy Yann-Erick, Roumanoff Patrick (1997) Verification of real-time systems using linear relation analysis. Formal Methods in System Design 11(2):157–185

    Article  Google Scholar 

  27. Higham NJ (1996) Accuracy and stability of numerical algorithms. Society for Industrial and Applied Mathematics, Philadelphia

    MATH  Google Scholar 

  28. IEEE Computer Society (2008) IEEE standard for floating-point arithmetic. In: IEEE Standard 754–2008

  29. Lyapunov AM (1947) Problème général de la stabilité du mouvement. Annals of Mathematics Studies 17. Princeton University Press, Princeton

    Google Scholar 

  30. Miné A (2001) The octagon abstract domain. In: AST 2001 in WCRE 2001, IEEE, pp 310–319. IEEE CS Press

  31. Miné A (2004) Relational abstract domains for the detection of floating-point run-time errors. In: ESOP, volume 2986 of LNCS, pp 3–17. Springer, http://www.di.ens.fr/~mine/publi/article-mine-esop04.pdf

  32. Monniaux D (2005) Compositional analysis of floating-point linear numerical filters. In: CAV, pp 199–212

  33. Roozbehani M, Féron E, Megretski A (2005) Modeling, optimization and computation for software verification. In: HSCC, pp 606–622

  34. Roux P (2013) Static analysis of control command systems: synthetizing non linear invariants. PhD thesis, Institut Supérieur de l’Aéronautique et de l’Espace

  35. Roux P, Garoche P-L (2013) Integrating policy iterations in abstract interpreters. In: Dang Van Hung and Mizuhito Ogawa, (eds), Automated technology for verification and analysis—11th international symposium, ATVA 2013, Hanoi, Vietnam, October 15–18, 2013. Proceedings, volume 8172 of Lecture Notes in Computer Science, pp 240–254. Springer

  36. Roux P, Garoche P-L (2014) Computing quadratic invariants with min- and max-policy iterations: a practical comparison. In: Jones CB, Pihlajasaari P, Sun J (eds), FM 2014: formal methods—19th international symposium, Singapore, May 12–16, 2014. Proceedings, volume 8442 of Lecture Notes in Computer Science, pp 563–578. Springer

  37. Roux P, Jobredeaux R, Garoche P-L, Féron E (2012) A generic ellipsoid abstract domain for linear time invariant systems. In: HSCC, pp 105–114

  38. Rump SM (2006) Verification of positive definiteness. BIT Numer Math 46:433–452

    Article  MATH  MathSciNet  Google Scholar 

  39. Rump SM (2010) Verification methods: Rigorous results using floating-point arithmetic. Acta Numer 19:287–449

    Article  MATH  MathSciNet  Google Scholar 

  40. Schrammel P, Jeannet B (2011) Logico-numerical abstract acceleration and application to the verification of data-flow programs. In: SAS, pp 233–248

  41. Seladji Y, Bouissou O (2013) Numerical abstract domain using support functions. NFM 7871:155–169

    Google Scholar 

  42. The Coq Development Team (2013) The Coq proof assistant reference manual, 2012. Version 8.4. Springer, Heidelberg

  43. Vandenberghe L, Boyd S (1996) Semidefinite programming. SIAM Rev 38(1):49–95

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

We would like to deeply thank the anonymous reviewers for their highly relevant comments to improve this paper. This work has been partially supported by the ANR-INSE-2012-007 Grant CAFEIN and the Aerospace Valley competitivity cluster.

Author information

Authors and Affiliations

  1. ISAE, University of Toulouse, Toulouse, France

    Pierre Roux

  2. ONERA - The French Aerospace Lab, Toulouse, France

    Pierre Roux & Pierre-Loïc Garoche

Authors
  1. Pierre Roux
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pierre-Loïc Garoche
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pierre-Loïc Garoche.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Roux, P., Garoche, PL. Practical policy iterations. Form Methods Syst Des 46, 163–196 (2015). https://doi.org/10.1007/s10703-015-0230-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10703-015-0230-7

Keywords

Search

Navigation