Abstract
Safety property (i.e., reachability) verification is undecidable for Turing-complete programming languages. Nonetheless, recent progress has lead to heuristic verification methods, such as the ones based on predicate abstraction with counterexample-guided refinement (CEGAR), that work surprisingly well in practice. In this paper, we investigate the effectiveness of the small refinement heuristic which, for abstraction refinement in CEGAR, uses (the predicates in) a small proof of the given counterexample’s spuriousness [3, 12, 17, 22]. Such a method has shown to be quite effective in practice but thus far lacked a theoretical backing. Our core result is that the heuristic guarantees certain bounds on the number of CEGAR iterations, relative to the size of a proof for the input program.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In this paper, a “proof” is a set of predicates. Note that this differs from the notion of proofs and proof sizes used in proof complexity [6].
- 2.
We do not impose \(F \vdash P \Leftrightarrow \mathsf {Abs}(P,F) = \mathsf {safe}\) to allow modeling verifiers whose abstraction process can prove more from the same predicates than the refinement process (cf. Sect. 3.1).
- 3.
In the literature, \(L_0\) is typically not treated as a loop, and a loop decomposition forms a forest.
- 4.
The decidability holds for any theory with effective interpolation (cf. the extended report [20]).
- 5.
In a sense, this paper poses and studies the question “assuming we have such algorithms for inferring small refinements, what can be said about the overall verification efficiency?”. Note that a possible outcome of the study can be a negative result; for example, showing that inferring small refinements is hard because otherwise it would give an efficient algorithm to some provably hard verification problem.
- 6.
It is easy to reduce any CFG program to an equivalent one whose unfoldings/paths would coincide with \(\mathsf {cex}^{ syn}\) (e.g., by encoding program locations in transition relation – cf. the extended report [20]). But, such a reduction is likely to affect the cost of the abstraction process and the refinement process.
References
International competition on software verification (SV-COMP). http://sv-comp.sosy-lab.org/
Aho, A.V., Sethi, R., Ullman, J.D.: Compilers: Principles, Techniques, and Tools. Addison-Wesley Longman Publishing Co., Inc., Boston (1986)
Albarghouthi, A., McMillan, K.L.: Beautiful interpolants. In: Sharygina and Veith [18], pp. 313–329
Ball, T., Rajamani, S.K.: The SLAM project: debugging system software via static analysis. In: Launchbury, J., Mitchell, J.C. (eds.) POPL, pp. 1–3. ACM (2002)
Beyer, D., Cimatti, A., Griggio, A., Keremoglu, M.E., Sebastiani, R.: Software model checking via large-block encoding. In: FMCAD, pp. 25–32. IEEE (2009)
Cook, S.A.: The complexity of theorem-proving procedures. In: Harrison, M.A., Banerji, R.B., Ullman, J.D. (eds.) STOC, pp. 151–158. ACM (1971)
Cousot, P., Cousot, R.: Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: Graham, R.M., Harrison, M.A., Sethi, R. (eds.), POPL, pp. 238–252. ACM (1977)
Esparza, J., Kiefer, S., Schwoon, S.: Abstraction refinement with Craig interpolation and symbolic pushdown systems. JSAT 5(1–4), 27–56 (2008)
Floyd, R.W.: Assigning meanings to programs. In: Symposia in Applied Mathematics, vol.19, pp. 19–32 (1967)
Gulavani, B.S., Chakraborty, S., Nori, A.V., Rajamani, S.K.: Refining abstract interpretations. Inf. Process. Lett. 110(16), 666–671 (2010)
Henzinger, T.A., Jhala, R., Majumdar, R., McMillan, K.L.: Abstractions from proofs. In: Jones, N.D., Leroy, X. (eds.) POPL, pp. 232–244. ACM (2004)
Hoder, K., Kovács, L., Voronkov, A.: Playing in the grey area of proofs. In: Field, J., Hicks, M. (eds.) POPL, pp. 259–272. ACM (2012)
Jhala, R., McMillan, K.L.: A practical and complete approach to predicate refinement. In: Hermanns, H., Palsberg, J. (eds.) TACAS 2006. LNCS, vol. 3920, pp. 459–473. Springer, Heidelberg (2006)
N. Kobayashi. Personal communication, 30 Aug (2012)
McMillan, K.L.: Quantified invariant generation using an interpolating saturation prover. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 413–427. Springer, Heidelberg (2008)
Rümmer, P., Hojjat, H., Kuncak, V.: Disjunctive interpolants for Horn-clause verification. In: Sharygina and Veith [18], pp. 347–363
Scholl, C., Pigorsch, F., Disch, S., Althaus, E.: Simple interpolants for linear arithmetic. In: DATE, pp. 1–6. IEEE (2014)
Sharygina, N., Veith, H. (eds.): CAV 2013. LNCS, vol. 8044. Springer, Heidelberg (2013)
Terauchi, T.: Dependent types from counterexamples. In: Hermenegildo, M.V., Palsberg, J. (eds.) POPL, pp. 119–130. ACM (2010)
Terauchi, T.: Explaining the effectiveness of small refinement heuristics in program verification with CEGAR (2015). http://www.jaist.ac.jp/terauchi
Terauchi, T., Unno, H.: Relaxed stratification: a new approach to practical complete predicate refinement. In: Vitek, J. (ed.) ESOP 2015. LNCS, vol. 9032, pp. 610–633. Springer, Heidelberg (2015)
Unno, H., Terauchi, T.: Inferring simple solutions to recursion-free horn clauses via sampling. In: Baier, C., Tinelli, C. (eds.) TACAS 2015. LNCS, vol. 9035, pp. 149–163. Springer, Heidelberg (2015)
Acknowledgements
We thank the anonymous reviewers for useful comments. This work was supported by MEXT Kakenhi 26330082 and 25280023, and JSPS Core-to-Core Program, A.Advanced Research Networks.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
A Proofs of the Main Results
A Proofs of the Main Results
1.1 A.1 Proof of Theorem 1
The following lemma states that each run of the refinement process returns a new proof.
Lemma 1
(Progress). Suppose \(\mathsf {Abs}(P,F)\) returns \(\pi \) and \(\mathsf {Ref}(\pi )\) returns \(F'\). Then, \(F' \not \subseteq F\).
Proof
Because \(\mathsf {Ref}(\pi )\) returns \(F'\), we have \(F' \vdash \pi \). Also, because \(\mathsf {Abs}(P,F)\) returns \(\pi \), we have \(\mathsf {Abs}(\pi ,F) \ne \mathsf {safe}\), and so \(F \not \vdash \pi \). Therefore, \(F \ne F'\), and by the monotonicity of \(\vdash \), it follows that \(F' \not \subseteq F\). Â Â Â \(\square \)
Theorem 1
Let the proof size metric be generic. Then, \(\textsc {CegVerif}_{\textsc {SR}}{}\) converges in at most \(\textit{exp}( mpfsize (P))\) many CEGAR iterations given a program P.
Proof
Let \(F_P\) be a proof of P such that \( mpfsize (P) = { size}(F_P)\). Then, for any counterexample \(\pi \) of P (i.e., \(\mathsf {Abs}(P,F')\) returns \(\pi \) for some \(F'\)), \(F_P \vdash \pi \), and therefore \( mpfsize (\pi ) \le { size}(F_P)\). There are at most \(c^{f( mpfsize (P))}\) proofs of size at most \(f({ size}(F_P))\). Therefore, by Lemma 1, \(\textsc {CegVerif}_{\textsc {SR}}\) converges in at most \(\textit{exp}( mpfsize (P))\) many CEGAR iterations.    \(\square \)
1.2 A.2 Proof of Theorem 10
Theorem
10. Let the proof size metric be syntactic, \(\vdash _{ cfg}\) be the proof relation, and \(\mathsf {Abs}^{ crt}\) be the abstraction process with \(\mathsf {cex}^{ syn}\) as the counterexample generator. Then, \(\textsc {CegVerif}_{\textsc {SR}}\) converges in \(\textit{poly}( mpfsize (P))^{{ maxhds}(P)}\) many CEGAR iterations given a CFG program P.
Proof
Let \(F_P\) be a proof of \(P = (G,T,v_{ ini},v_{ err})\) such that \( mpfsize (P) = { size}(F_P)\). Then, for any counterexample \(\pi \) of P (i.e., \(\mathsf {Abs}^{ crt}(P,F)\) returns \(\pi \) for some F), \(F_P \vdash \pi \), and so \( mpfsize (\pi ) \le { size}(F_P)\). Let \({ lim} = (f({ size}(F_P))+2)^{{ maxhds}(P)}+1\) (“\(+2\)” accounts for \(\{{\bot ,\top }\}\)). Let \(\pi _{ lim}\) be a counterexample in \(\mathsf {cex}^{ syn}(P)\) obtained by unfolding the loops at least \({ lim}\) many times. We show that for any F such that \(F \vdash _{ cfg}\pi _{ lim}\) and \({ size}(F) \le f({ size}(F_P))\), \(F \vdash _{ crt}P\). (We call such a counterexample \(\pi _{ lim}\) sufficiently large.) Then, the result follows from the fact that there are only \({ lim}\) many counterexamples in \(\mathsf {cex}^{ syn}(P)\) that are obtained by unfolding the loops at most \({ lim}\) many times, and the fact that no counterexample is returned more than once by the refinement process in a run of \(\textsc {CegVerif}_{\textsc {SR}}\).
Let \(\pi _{ lim}= (G_{ lim},T_{ lim},v_{ ini},v_{ err})\). Let \(\sigma _{ lim}: v (G_{ lim})\rightarrow F\cup \{{\bot ,\top }\}\) be such that \({ size}(F) \le f({ size}(F_P))\) and \(\sigma _{ lim}\vdash _{ cfg}\pi _{ lim}\). Let \(k \ge { lim}\) be the number of times the loops are unfolded in \(\pi _{ lim}\). We construct \(\sigma : v (G)\rightarrow F^{\wedge \vee }\) such that \(\sigma \vdash _{ crt}^F P\) by “folding” the unfolded loops in a bottom up manner. We initialize \(\sigma = \sigma _{ lim}\), and \(\gamma = \pi _{ lim}\). We iteratively fold \(\gamma \) from leaf loops, while maintaining the property that \(\sigma \vdash _{ crt}^F \gamma \). Then, the result follows from the fact that \(\gamma \) becomes P at the root of the folding process.
Let \(\gamma \) and \(\sigma \) be the current CFG and its Cartesian predicate abstraction node-wise inductive invariant (i.e., \(\sigma \vdash _{ crt}^F \gamma \)). Let \(\gamma = C[t_1' \cup t_2' \cup \dots t_k' \cup {t_k}\mathord \setminus \{{\varvec{h}_k}\})]\) where \(\varvec{h}_0 = \varvec{h}\), \(t_0 = t\), \(\varvec{h}_{i+1} = `(\varvec{h}_i)\), \({t_i}' = t_i\langle { v (t_i)\mathord \setminus \{{\varvec{h}_i}\}, sc (t_i)}\rangle \), and \(t_{i+1} = t_i\langle {\varvec{h}_i,\varvec{h}_i}\rangle \) for each \(i \in \{{0,\dots ,k-1}\}\). Let \(\gamma ' = C[{ loop}\;({\varvec{h}})\;{t}]\). That is, \(\gamma '\) is obtained by folding the unfolded loop of \(\gamma \). We construct \(\sigma ': v (\gamma ')\rightarrow F^{\wedge \vee }\) such that \(\sigma ' \vdash _{ crt}^F \gamma '\) as follows. Because \(\mathop {\textit{ran}}\nolimits (\sigma _{lim}) = F\cup \{{\bot ,\top }\}\), for some \(1 \le i_1 < i_2 \le (f({ size}(F_P))+2)^{|\varvec{h}|}+1 \le k\), \(\sigma _{ lim}(h^{i_1}) = \sigma _{ lim}(h^{i_2})\) for each \(h \in \{{\varvec{h}}\}\). By construction, \(\sigma _{ lim}(h^i) = \sigma (h^i)\) for each \(i \in \{{1,\dots ,k}\}\) and \(h \in \{{\varvec{h}}\}\). We set \(\sigma '(v) = \bigvee _{i=1}^{i_2} \sigma (v^i)\) for each \(v \in sc (t)\), and \(\sigma '(v) = \sigma (v)\) for each \(v \in v (\gamma )\setminus sc (t)\). Then, it can be seen that \(\sigma ' \vdash _{ crt}^F \gamma '\). Â Â Â \(\square \)
1.3 A.3 Proof of Theorem 11
Theorem
11. Let the proof size metric be generic, \(\vdash _{ cfg}\) be the proof relation, and \(\mathsf {Abs}^{ crt}\) be the abstraction process with \(\mathsf {cex}^{ syn}\) as the counterexample generator. Then, there exists a CFG program P with a proof \(F \vdash _{ cfg}P\) on which \(\textsc {CegVerif}_{\textsc {SR}}\) may take \(\textit{exp}({ size}(F))\) many CEGAR iterations to converge.
Proof
We show that \(P_{ ex}\) from Fig. 4 is such a program. As remarked before, we have \(F_{ inv} \vdash _{ cfg}P_{ ex}\) where \(F_{ inv} = \{a = b \Rightarrow y = x + z\}\). For each \(k > 0\), let \(\pi _k \in \mathsf {cex}^{ syn}(P_{ ex})\) be the counterexample obtained by unfolding each loop k times, and let \(F_k = \{{\bigvee F \mid F \subseteq \{{\phi _i \mid 0 \le i \le k}\}}\}\) where \(\phi _i \equiv x = a \wedge y=b+i \wedge z=i\). Then, \(F_k \vdash _{ cfg}\pi _k\) for each \(k > 0\), and \(\bigcup _{i=1}^j F_i \not \vdash _{ crt}\pi _k\) for each \(k > j > 0\).
Let \(\ell > 0\). Define \({ size}\) as follows: \({ size}(F_{ inv}) = \ell \), \({ size}(F_k) = \lfloor \log k\rfloor \) for \(k \in \{{1,\dots ,2^\ell -1}\}\), and \({ size}(F') = \ell +{ syntactic}(F')\) for \(F' \in \mathcal {P}(\mathcal {T})\setminus (\{{F_{ inv}}\} \cup \{{F_k \mid 1 \le k \le 2^\ell -1}\})\) where \({ syntactic}(F')\) is the syntactic size of \(F'\). Note that \({ size}\) is a generic proof size metric. Then, \(\textsc {CegVerif}_{\textsc {SR}}\) takes \(2^{{ size}(F_{ inv})}\) iterations to converge when \(\mathsf {Abs}^{ crt}\) returns the counterexamples \(\pi _1,...\pi _{2^\ell -1}\) in the first \(2^\ell -1\) iterations and \(\mathsf {Ref}\) returns the proofs \(F_1,\dots ,F_{2^\ell -1}\) before returning \(F_{ inv}\). Â Â Â \(\square \)
Rights and permissions
Copyright information
© 2015 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Terauchi, T. (2015). Explaining the Effectiveness of Small Refinement Heuristics in Program Verification with CEGAR. In: Blazy, S., Jensen, T. (eds) Static Analysis. SAS 2015. Lecture Notes in Computer Science(), vol 9291. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-48288-9_8
Download citation
DOI: https://doi.org/10.1007/978-3-662-48288-9_8
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-48287-2
Online ISBN: 978-3-662-48288-9
eBook Packages: Computer ScienceComputer Science (R0)