Rongchen Xu
ABSTRACT
Loop invariant generation has long been a challenging problem. Black-box learning has recently emerged as a promising method for inferring loop invariants. However, the performance depends heavily on the quality of collected examples. In many cases, only after tens or even hundreds of constraint queries, can a feasible invariant be successfully inferred.
To reduce the gigantic number of constraint queries and improve the performance of black-box learning, we introduce interval counterexamples into the learning framework. Each interval counterexample represents a set of counterexamples from constraint solvers. We propose three different generalization techniques to compute interval counterexamples. The existing decision tree algorithm is also improved to adapt interval counterexamples. We evaluate our techniques and report over 40% improvement on learning rounds and verification time over the state-of-the-art approach.
Supplemental Material
- Muqsit Azeem, Kumar Madhukar, and R Venkatesh. 2018. Generalizing SpecificInstance Interpolation Proofs with SyGuS. In Proceedings of the 40th International Conference on Software Engineering : New Ideas and Emerging Results (Gothenburg, Sweden) (ICSE-NIER '18). Association for Computing Machinery, New York, NY, USA, 57-60. https://doi.org/10.1145/3183399.3183412 Google Scholar
Digital Library
- Thomas Ball and Sriram K. Rajamani. 2001. The SLAM Toolkit. In Computer Aided Verification, Gérard Berry, Hubert Comon, and Alain Finkel (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 260-264. https://doi.org/10.1007/3-540-44585-4_25 Google ScholarCross Ref
- Clark Barrett and Cesare Tinelli. 2018. Satisfiability Modulo Theories. In Handbook of Model Checking, Edmund M. Clarke, Thomas A. Henzinger, Helmut Veith, and Roderick Bloem (Eds.). Springer International Publishing, Cham, 305-343. https://doi.org/10.1007/978-3-319-10575-8_11 Google ScholarCross Ref
- Cristiano Calcagno, Dino Distefano, and Viktor Vafeiadis. 2009. Bi-abductive Resource Invariant Synthesis. In Programming Languages and Systems, Zhenjiang Hu (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 259-274. https://doi. org/10.1007/11817963_14 Google ScholarDigital Library
- Kwonsoo Chae, Hakjoo Oh, Kihong Heo, and Hongseok Yang. 2017. Automatically Generating Features for Learning Program Analysis Heuristics for C-like Languages. Proc. ACM Program. Lang. 1, OOPSLA, Article 101 (Oct. 2017 ), 25 pages. https://doi.org/10.1145/3133925 Google ScholarDigital Library
- Edmund Clarke, Orna Grumberg, Somesh Jha, Yuan Lu, and Helmut Veith. 2003. Counterexample-Guided Abstraction Refinement for Symbolic Model Checking. J. ACM 50, 5 (Sept. 2003 ), 752-794. https://doi.org/10.1145/876638.876643 Google ScholarDigital Library
- Patrick Cousot and Radhia Cousot. 1979. Systematic Design of Program Analysis Frameworks. In Proceedings of the 6th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages (San Antonio, Texas) ( POPL '79). Association for Computing Machinery, New York, NY, USA, 269-282. https://doi.org/10. 1145/567752.567778 Google ScholarDigital Library
- Patrick Cousot and Nicolas Halbwachs. 1978. Automatic Discovery of Linear Restraints among Variables of a Program. In Proceedings of the 5th ACM SIGACTSIGPLAN Symposium on Principles of Programming Languages (Tucson, Arizona) (POPL '78). Association for Computing Machinery, New York, NY, USA, 84-96. https://doi.org/10.1145/512760.512770 Google ScholarDigital Library
- Leonardo de Moura and Nikolaj Bjørner. 2008. Z3: An Eficient SMT Solver. In Tools and Algorithms for the Construction and Analysis of Systems, C. R. Ramakrishnan and Jakob Rehof (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 337-340. https://doi.org/10.1007/978-3-540-78800-3_24 Google ScholarCross Ref
- Isil Dillig, Thomas Dillig, Boyang Li, and Ken McMillan. 2013. Inductive Invariant Generation via Abductive Inference. SIGPLAN Not. 48, 10 (Oct. 2013 ), 443-456. https://doi.org/10.1145/2544173.2509511 Google ScholarDigital Library
- Michael D. Ernst, Adam Czeisler, William G. Griswold, and David Notkin. 2000. Quickly Detecting Relevant Program Invariants. In Proceedings of the 22nd International Conference on Software Engineering (Limerick, Ireland) (ICSE '00). Association for Computing Machinery, New York, NY, USA, 449-458. https: //doi.org/10.1145/337180.337240 Google ScholarDigital Library
- P. Ezudheen, Daniel Neider, Deepak D'Souza, Pranav Garg, and P. Madhusudan. 2018. Horn-ICE Learning for Synthesizing Invariants and Contracts. Proc. ACM Program. Lang. 2, OOPSLA, Article 131 (Oct. 2018 ), 25 pages. https://doi.org/10. 1145/3276501 Google ScholarDigital Library
- G. Fedyukovich, S. J. Kaufman, and R. Bodík. 2017. Sampling invariants from frequency distributions. In 2017 Formal Methods in Computer Aided Design (FMCAD). 100-107. https://doi.org/10.23919/FMCAD. 2017.8102247 Google ScholarCross Ref
- Cormac Flanagan and K. Rustan M. Leino. 2001. Houdini, an Annotation Assistant for ESC/Java. In FME 2001: Formal Methods for Increasing Software Productivity, José Nuno Oliveira and Pamela Zave (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 500-517. https://doi.org/10.1007/3-540-45251-6_29 Google ScholarCross Ref
- Pranav Garg, Christof Löding, P. Madhusudan, and Daniel Neider. 2014. ICE: A Robust Framework for Learning Invariants. In Computer Aided Verification, Armin Biere and Roderick Bloem (Eds.). Springer International Publishing, Cham, 69-87. https://doi.org/10.1007/978-3-319-08867-9_5 Google ScholarDigital Library
- Pranav Garg, Daniel Neider, P. Madhusudan, and Dan Roth. 2016. Learning Invariants Using Decision Trees and Implication Counterexamples. SIGPLAN Not. 51, 1 (Jan. 2016 ), 499-512. https://doi.org/10.1145/2914770.2837664 Google ScholarDigital Library
- Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, and Grégoire Sutre. 2003. Software Verification with BLAST. In Model Checking Software, Thomas Ball and Sriram K. Rajamani (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 235-239. https://doi.org/10.1007/3-540-44829-2_17 Google ScholarCross Ref
- Ranjit Jhala and K. L. McMillan. 2006. A Practical and Complete Approach to Predicate Refinement. In Tools and Algorithms for the Construction and Analysis of Systems, Holger Hermanns and Jens Palsberg (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 459-473. https://doi.org/10.1007/11691372_33 Google ScholarDigital Library
- Michael Karr. 1976. Afine relationships among variables of a program. Acta Informatica 6, 2 (jun 1976 ), 133-151. https://doi.org/10.1007/BF00268497 Google ScholarDigital Library
- Siddharth Krishna, Christian Puhrsch, and Thomas Wies. 2015. Learning Invariants using Decision Trees. CoRR abs/1501.04725 ( 2015 ), 15. arXiv: 1501.04725 http://arxiv.org/abs/1501.04725Google Scholar
- J. Li, J. Sun, L. Li, Q. L. Le, and S. Lin. 2017. Automatic loop-invariant generation anc refinement through selective sampling. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). 782-792. https://doi.org/ 10.1109/ASE. 2017.8115689 Google ScholarCross Ref
- S. Lin, J. Sun, H. Xiao, Y. Liu, D. Sanán, and H. Hansen. 2017. FiB: Squeezing loop invariants by interpolation between forward/backward predicate transformers. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). 793-803. https://doi.org/10.1109/ASE. 2017.8115690 Google ScholarCross Ref
- Kenneth L McMillan. 2003. Interpolation and SAT-based model checking. In International Conference on Computer Aided Verification. Springer, 1-13. https: //doi.org/10.1007/978-3-540-45069-6_1 Google ScholarCross Ref
- Kenneth L. McMillan. 2006. Lazy Abstraction with Interpolants. In Computer Aided Verification, Thomas Ball and Robert B. Jones (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 123-136. https://doi.org/10.1007/11817963_14 Google ScholarDigital Library
- Kenneth L. McMillan. 2010. Lazy Annotation for Program Testing and Verification. In Computer Aided Verification, Tayssir Touili, Byron Cook, and Paul Jackson (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 104-118. https://doi.org/ 10.1007/978-3-642-14295-6_10 Google ScholarDigital Library
- Antoine Miné. 2006. The octagon abstract domain. Higher-Order and Symbolic Computation 19, 1 (mar 2006 ), 31-100. https://doi.org/10.1007/s10990-006-8609-1 Google ScholarDigital Library
- Saswat Padhi, Rahul Sharma, and Todd Millstein. 2016. Data-Driven Precondition Inference with Learned Features. SIGPLAN Not. 51, 6 ( June 2016 ), 42-56. https: //doi.org/10.1145/2980983.2908099 Google ScholarDigital Library
- Oded Padon, Kenneth L. McMillan, Aurojit Panda, Mooly Sagiv, and Sharon Shoham. 2016. Ivy: Safety Verification by Interactive Generalization. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (Santa Barbara, CA, USA) ( PLDI '16). Association for Computing Machinery, New York, NY, USA, 614-630. https://doi.org/10.1145/2908080. 2908118 Google ScholarDigital Library
- J. Ross Quinlan. 1986. Induction of decision trees. Machine learning 1, 1 (mar 1986 ), 81-106. https://doi.org/10.1007/BF00116251 Google ScholarCross Ref
- J. Ross Quinlan. 1993. C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. https://doi.org/10.1007/BF00993309 Google ScholarCross Ref
- Lior Rokach and Oded Z Maimon. 2008. Data mining with decision trees: theory and applications. Vol. 69. World scientific. https://doi.org/10.1142/9789812771728_0001 Google ScholarCross Ref
- S Rasoul Safavian and David Landgrebe. 1991. A survey of decision tree classifier methodology. IEEE Transactions on Systems, Man, and Cybernetics 21, 3 ( 1991 ), 660-674. https://doi.org/10.1109/21.97458 Google ScholarCross Ref
- C. E. Shannon. 1948. A mathematical theory of communication. The Bell System Technical Journal 27, 3 ( July 1948 ), 379-423. https://doi.org/10.1002/j.1538-7305. 1948.tb01338.x Google ScholarCross Ref
- Rahul Sharma and Alex Aiken. 2016. From invariant checking to invariant inference using randomized search. Formal Methods in System Design 48, 3 (jun 2016 ), 235-256. https://doi.org/10.1007/s10703-016-0248-5 Google ScholarDigital Library
- Rahul Sharma, Saurabh Gupta, Bharath Hariharan, Alex Aiken, Percy Liang, and Aditya V. Nori. 2013. A Data Driven Approach for Algebraic Loop Invariants. In Programming Languages and Systems, Matthias Felleisen and Philippa Gardner (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 574-592. https://doi.org/ 10.1007/978-3-642-37036-6_31 Google ScholarDigital Library
- Rahul Sharma, Saurabh Gupta, Bharath Hariharan, Alex Aiken, and Aditya V. Nori. 2013. Verification as Learning Geometric Concepts. In Static Analysis, Francesco Logozzo and Manuel Fähndrich (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 388-411. https://doi.org/10.1007/978-3-642-38856-9_21 Google ScholarCross Ref
- Rahul Sharma, Aditya V. Nori, and Alex Aiken. 2012. Interpolants as Classifiers. In Computer Aided Verification, P. Madhusudan and Sanjit A. Seshia (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 71-87. https://doi.org/10.1007/978-3-642-31424-7_11 Google ScholarDigital Library
- Xujie Si, Hanjun Dai, Mukund Raghothaman, Mayur Naik, and Le Song. 2018. Learning Loop Invariants for Program Verification. In Advances in Neural Information Processing Systems 31, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.). Curran Associates, Inc., 7751-7762. https://doi.org/10.5555/3327757.3327873 Google ScholarDigital Library
- He Zhu, Stephen Magill, and Suresh Jagannathan. 2018. A Data-Driven CHC Solver. SIGPLAN Not. 53, 4 ( June 2018 ), 707-721. https://doi.org/10.1145/3296979. 3192416 Google ScholarDigital Library
- He Zhu, Aditya V. Nori, and Suresh Jagannathan. 2015. Learning Refinement Types. SIGPLAN Not. 50, 9 (Aug. 2015 ), 400-411. https://doi.org/10.1145/2858949. 2784766 Google ScholarDigital Library
Index Terms
- Interval counterexamples for loop invariant learning
Recommendations
Program analysis as constraint solving
PLDI '08A constraint-based approach to invariant generation in programs translates a program into constraints that are solved using off-the-shelf constraint solvers to yield desired program invariants.
In this paper we show how the constraint-based approach can ...
Loop Invariant Inference through SMT Solving Enhanced Reinforcement Learning
ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and AnalysisInferring loop invariants is one of the most challenging problems in program verification. It is highly desired to incorporate machine learning when inferring. This paper presents a Reinforcement Learning (RL) pruning framework to infer loop ...
Verification and refutation of C programs based on k-induction and invariant inference
AbstractDepthK is a source-to-source transformation tool that employs bounded model checking (BMC) to verify and falsify safety properties in single- and multi-threaded C programs, without manual annotation of loop invariants. Here, we describe and ...
Comments