Skip to main content
SpringerLink
Log in

A framework for counterexample generation and exploration

  • Special section FASE'04/05
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

Model-checking is becoming an accepted technique for debugging hardware and software systems. Debugging is based on the “Check/Analyze/Fix” loop: check the system against a desired property, producing a counterexample when the property fails to hold; analyze the generated counterexample to locate the source of the error; fix the flawed artifact—the property or the model. The success of model-checking non-trivial systems critically depends on making this Check/Analyze/Fix loop as tight as possible. In this paper, we concentrate on the Analyze part of the debugging loop. To this end, we present a framework for generating, structuring and exploring counterexamples, implemented in a tool called KEGVis. The framework is based on the idea that the most general type of evidence to why a property holds or fails to hold is a proof. Such proofs can be presented to the user in the form of proof-like counterexamples, without sacrificing any of the intuitiveness and close relation to the model that users have learned to expect from model-checkers. Moreover, proof generation is flexible, and can be controlled by strategies, whether built into the tool or specified by the user, thus enabling generation of the most “interesting” counterexample and its interactive exploration. Moreover, proofs can be used to generate and display all relevant evidence together, a technique referred to as abstract counterexamples. Overall, our framework can be used for explaining the reason why the property failed or succeeded, determining whether the property was correct (“specification debugging”), and for general model exploration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ball T., Podelski A. and Rajamani S. (2003). Boolean and Cartesian abstraction for model checking C programs. Int. J. Softw. Tools Technol. Transf. (STTT) 5(1): 49–58

    Article  Google Scholar 

  2. Barner, S., Ben-David, S., Gringauze, A., Sterin, B., Wolfsthal, Y.: An algorithmic approach to design exploration. In: Proceedings of Formal Methods Europe (FME’02), vol. 2391 of LNCS, pp. 146–162. Springer, Copenhagen, Denmark (2002)

  3. Chan, W.: Temporal-logic queries. In: Proceedings of the 12th conference on Computer Aided Verification (CAV’00), vol. 1855 of LNCS, pp. 450–463. Springer, Chicago, IL, USA (2000)

  4. Chechik, M., Devereux, B., Gurfinkel, A.: χChek: a multi-valued model-checker. In: Proceedings of 14th International Conference on Computer-Aided Verification (CAV’02), vol. 2404 of LNCS, pp. 505–509, Springer, Copenhagen, Denmark (2002)

  5. Chechik, M., Gurfinkel, A.: A framework for counterexample generation and exploration. In: Proceedings of Formal Aspects in Software Engineering (FASE’05), vol. 3442 of LNCS, pp. 217–233. Springer, Heidelberg (2005)

  6. Cimatti, A., Clarke, E.M., Giunchiglia, F., Roveri, M.: NuSMV: a new Symbolic Model Verifier. In: Halbwachs, N., Peled, D. (eds.) Proceedings of 11th Conference on Computer-Aided Verification (CAV’99), vol. 1633 of LNCS, pp. 495–499 Springer, Trento, Italy (1999)

  7. Clarke E., Grumberg O. and Peled D. (1999). Model Checking. MIT Press, Cambridge

    Google Scholar 

  8. Clarke, E.M., Grumberg, O., McMillan, K.L., Zhao, X.: Efficient generation of counterexamples and witnesses in symbolic model checking. In: Proceedings of 32nd Design Automation Conference (DAC 95), pp. 427–432, San Francisco, CA, USA (1995)

  9. Clarke, E.M., Lu, Y., Jha, S., Veith, H.: Tree-like counterexamples in model checking. In: Proceedings of the 17th Annual IEEE Symposium on Logic in Computer Science (LICS’02), pp. 19–29. IEEE Computer Society Copenhagen, Denmark (2002)

  10. Copty F., Irron A., Weissberg O., Kropp N. and Kamhi G. (2003). Efficient debugging in a formal verification environment. Int. J. Softw. Tools Technol. Transf. (STTT) 4(3): 335–348

    Article  Google Scholar 

  11. Dong, Y., Ramakrishnan, C.R., Smolka, S.A.: Evidence explorer: a tool for exploring model-checking proofs. In: Proceedings of the 15th International Conference on Computer-Aided Verification (CAV’03), vol. 2725 of LNCS, pp. 215–218 (2003)

  12. Dong, Y., Ramakrishnan, C.R., Smolka, S.A.: Model checking and evidence exploration. In: Proceedings of the 10th IEEE International Conference and Workshop on the Engineering of Computer Based Systems (ECBS’03), pp. 214–223, Alabama, USA (2003)

  13. Dwyer, M., Avrunin, G., Corbett, J.: Patterns in property specifications for finite-state verification. In: Proceedings of 21st International Conference on Software Engineering (ICSE’99) (1999)

  14. Groce, A., Visser, W.: What went wrong: explaining counterexamples. In: Proceedings of SPIN Workshop on Model Checking of Software, pp. 121–135 (2003)

  15. Gurfinkel, A.: Multi-valued symbolic model-checking: fairness, counter-examples, running time. Master’s Thesis, University of Toronto, Department of Computer Science (2002)

  16. Gurfinkel, A., Chechik, M.: Proof-like Counterexamples. In: Proceedings of 9th International Conference on Tools and for the Construction and Analysis of Systems (TACAS’03), vol. 2619 of LNCS, pp. 160–175, Springer Warsaw, Poland (2003)

  17. Gurfinkel A., Chechik M. and Devereux B. (2003). Temporal logic query checking: a tool for model exploration. IEEE Trans. Softw. Eng. 29(10): 898–914

    Article  Google Scholar 

  18. Laroussinie F. (1995). About the expressive power of CTL combinators. Inf. Process. Lett. 54: 343–345

    Article  MATH  MathSciNet  Google Scholar 

  19. Namjoshi, K.: Certifying model checkers. In: Proceedings of 13th International Conference on Computer-Aided Verification (CAV’01), vol. 2102 of LNCS. Springer, Heidelberg (2001)

  20. Owre, S., Shankar, N., Rushby, J.: User guide for the PVS specification and verification system (draft). In: Technical report, Computer Science Lab, SRI International, Menlo Park, CA (1993)

  21. Peled, D., Pnueli, A., Zuck, L.: From falsification to verification. In: Proceedings of 21st Conference on the Foundations of Software Technology and Theoretical Computer Science (FST&TCS’01), vol. 2245 of LNCS, pp. 292–304. Springer, Heidelberg (2001)

  22. Peled, D., Zuck, L.: From model checking to a temporal proof. In: Proceedings of the 8th International SPIN Workshop (SPIN’2001), vol. 2057 of LNCS, pp. 1–14. Springer, Toronto, Canada (2001)

  23. Plath M.C. and Ryan M.D. (1999). SFI: a feature integration tool. In: Berghammer, R. and Lakhnech, Y. (eds) Tool Support for System Specification, Development and Verification. Advances in Computer Science, pp 201–216. Springer, Heidelberg

    Google Scholar 

  24. Stirling, C., Walker, D.: Local model-checking in the modal mu-calculus. Theoret. Comput. Sci. 89, (1991)

  25. Tan, L., Cleaveland, R.: Evidence-based model checking. In: Proceedings of 14th Conference on Computer-Aided Verification (CAV’02), vol. 2404 of LNCS, pp. 455–470. Springer Copenhagen, Denmark (2002)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Toronto, Toronto, ON, M5S 3G4, Canada

    Marsha Chechik

  2. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    Arie Gurfinkel

Authors
  1. Marsha Chechik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arie Gurfinkel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marsha Chechik.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chechik, M., Gurfinkel, A. A framework for counterexample generation and exploration. Int J Softw Tools Technol Transf 9, 429–445 (2007). https://doi.org/10.1007/s10009-007-0047-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-007-0047-9

Keywords

Search

Navigation