ABSTRACT
Two popular forms of dynamic analysis, random testing and explicit-state software model checking, are perhaps best viewed as search strategies for exploring the state spaces introduced by nondeterminism in program inputs. We present an approach that enables this nondeterminism to be expressed in the SPIN model checker's PROMELA language, and then lets users generate either model checkers or random testers from a single harness for a tested C program. Our approach makes it easy to compare model checking and random testing for models with precisely the same input ranges and probabilities and allows us to mix random testing with model checking's exhaustive exploration of non-determinism. The PROMELA language, as intended in its design, serves as a convenient notation for expressing nondeterminism and mixing random choices with nondeterministic choices. We present and discuss a comparison of random testing and model checking. The results derive from using our framework to test a C program with an effectively infinite state space, a module in JPL's next Mars rover mission. More generally, we show how the ability of the SPIN model checker to call C code can be used to extend SPIN's features, and hope to inspire others to use the same methods to implement dynamic analyses that can make use of efficient state storage, matching, and backtracking.
- http://mars.jpl.nasa.gov/msl/.Google Scholar
- Edmund M. Clarke, Orna Grumberg, and Doron Peled. Model Checking. MIT Press, 2000.Google Scholar
- Matthew B. Dwyer, Sebastian G. Elbaum, Suzette Person, and Ragul Purandare. Parallel randomized state-space search. In International Conference on Software Engineering, pages 3--12, 2007. Google ScholarDigital Library
- Matthew B. Dwyer, Suzette Person, and Sebastian Elbaum. Controlling factors in evaluating path-sensitive error detection techniques. In Foundations of Software Engineering, pages 92--104, 2006. Google ScholarDigital Library
- Michael Ernst, Jake Cockrell, William Griswold, and David Notkin. Dynamically discovering likely program invariants to support program evolution. In International Conference on Software Engineering, pages 213--224, 1999. Google ScholarDigital Library
- Alex Groce, Gerard Holzmann, and Rajeev Joshi. Randomized differential testing as a prelude to formal verification. In International Conference on Software Engineering, pages 621--631, 2007. Google ScholarDigital Library
- Alex Groce and Rajeev Joshi. Extending model checking with dynamic analysis. In International Conference on Verification, Model Checking, and Abstract Interpretation, pages 142--156, 2008. Google ScholarDigital Library
- Alex Groce and Willem Visser. Heuristics for model checking Java programs. Software Tools for Technology Transfer, 6(4): 260--276, 2004.Google ScholarDigital Library
- Richard Hamlet. Random testing. In Encyclopedia of Software Engineering, pages 970--978. Wiley, 1994.Google Scholar
- Richard Hamlet. When only random testing will do. In International Workshop on Random Testing, pages 1--9, 2006. Google ScholarDigital Library
- Gerard Holzmann and Rajeev Joshi. Model-driven software verification. In SPIN Workshop on Model Checking of Software, pages 76--91, 2004.Google ScholarCross Ref
- Gerard J. Holzmann. The SPIN Model Checker: Primer and Reference Manual. Addison-Wesley Professional, 2003. Google ScholarDigital Library
- William McKeeman. Differential testing for software. Digital Technical Journal of Digital Equipment Corporation, 10(1): 100--107, 1998.Google Scholar
- Carlos Pacheco, Shuvendu K. Lahiri, Michael D. Ernst, and Thomas Ball. Feedback-directed random test generation. In International Conference on Software Engineering, pages 75--84, 2007. Google ScholarDigital Library
- Willem Visser, Klaus Havelund, Guillaume Brat, SeungJoon Park, and Flavio Lerda. Model checking programs. Automated Software Engineering, 10(2): 203--232, April 2003. Google ScholarDigital Library
- Willem Visser, Corina Pǎsǎreanu, and Radek Pelanek. Test input generation for Java containers using state matching. In International Symposium on Software Testing and Analysis, pages 37--48, 2006. Google ScholarDigital Library
Index Terms
- Random testing and model checking: building a common framework for nondeterministic exploration
Recommendations
Bounded model checking of high-integrity software
HILT '13: Proceedings of the 2013 ACM SIGAda annual conference on High integrity language technologyModel checking [5] is an automated algorithmic technique for exhaustive verification of systems, described as finite state machines, against temporal logic [9] specifications. It has been used successfully to verify hardware at an industrial scale [6]. ...
Conditional model checking: a technique to pass information between verifiers
FSE '12: Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software EngineeringSoftware model checking, as an undecidable problem, has three possible outcomes: (1) the program satisfies the specification, (2) the program does not satisfy the specification, and (3) the model checker fails. The third outcome usually manifests itself ...
Regular model checking for LTL(MSO)
Regular model checking is a form of symbolic model checking for parameterized and infinite-state systems whose states can be represented as words of arbitrary length over a finite alphabet, in which regular sets of words are used to represent sets of ...
Comments