Pallavi Joshi
ABSTRACT
Testing with manually generated test cases often results in poor coverage and fails to discover many corner case bugs and security vulnerabilities. Automated test generation techniques based on static or symbolic analysis usually do not scale beyond small program units. We propose predictive testing, a new method for amplifying the effectiveness of existing test cases using symbolic analysis. We assume that a software system has an associated test suite consisting of a set of test inputs and a set of program invariants, in the form of a set of assert statements that the software must always satisfy. Predictive testing uses a combination of concrete and symbolic execution, similar to concolic execution, on the provided test inputs to discover if any of the assertions encountered along a test execution path could be violated for some closely related inputs. We extend predictive testing to catch bugs related to memory-safety violations, integer overflows, and string-related vulnerabilities. Furthermore, we propose a novel technique that leverages the results of unit testing to hoist assertions located deep inside the body of a unit function to the beginning of the unit function. This enables predictive testing to encounter assertions more often in test executions and thereby significantly amplifies the effectiveness of testing.
- D. Beyer, A. J. Chlipala, T. A. Henzinger, R. Jhala, and R. Majumdar. Generating Test from Counterexamples. In Proc. of the 26th ICSE, pages 326--335, 2004. Google Scholar
Digital Library
- D. Bird and C. Munoz. Automatic Generation of Random Self-Checking Test Cases. IBM Systems Journal, 22(3):229--245, 1983. Google ScholarDigital Library
- J. Condit, M. Harren, Z. Anderson, D. Gay, and G. Necula. Dependent types for low-level programs. Technical Report EECS-2006-129, UC Berkeley, 2006.Google Scholar
- B. Dutertre and L. M. de Moura. A fast linear-arithmetic solver for dpll(t). In Computer Aided Verification, volume 4144 of LNCS, pages 81--94, 2006. Google ScholarDigital Library
- J. E. Forrester and B. P. Miller. An Empirical Study of the Robustness of Windows NT Applications Using Random Testing. In Proceedings of the 4th USENIX Windows System Symposium, 2000. Google ScholarDigital Library
- P. Godefroid, N. Klarlund, and K. Sen. DART: Directed automated random testing. In Proc. of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation (PLDI), 2005. Google ScholarDigital Library
- R. W. M. Jones and P. H. J. Kelly. Backwards-compatible bounds checking for arrays and pointers in C programs. AADEBUG, 1997.Google Scholar
- S. Khurshid, C. S. Pasareanu, and W. Visser. Generalized symbolic execution for model checking and testing. In Proc. 9th Int. Conf. on TACAS, pages 553--568, 2003. Google ScholarDigital Library
- J. C. King. Symbolic Execution and Program Testing. Communications of the ACM, 19(7):385--394, 1976. Google ScholarDigital Library
- R. Majumdar and K. Sen. Hybrid concolic testing. In 29th International Conference on Software Engineering (ICSE'07). IEEE. (To Appear). Google ScholarDigital Library
- G. C. Necula, J. Condit, M. Harren, S. McPeak, and W. Weimer. CCured: Type-safe retrofitting of legacy software. TOPLAS, 27(3), May 2005. Google ScholarDigital Library
- G. C. Necula, S. McPeak, S. P. Rahul, and W. Weimer. CIL: Intermediate Language and Tools for Analysis and transformation of C Programs. In Proceedings of Conference on compiler Construction, pages 213--228, 2002. Google ScholarDigital Library
- J. Offut and J. Hayes. A Semantic Model of Program Faults. In Proc. of ISSTA'96, pages 195--200, 1996. Google ScholarDigital Library
- K. Sen, D. Marinov, and G. Agha. CUTE: A concolic unit testing engine for C. In 5th joint meeting of the European Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE'05). ACM, 2005. Google ScholarDigital Library
Index Terms
- Predictive testing: amplifying the effectiveness of software testing
Recommendations
Concolic testing
ASE '07: Proceedings of the 22nd IEEE/ACM International Conference on Automated Software EngineeringConcolic testing automates test input generation by combining the concrete and symbolic (concolic) execution of the code under test. Traditional test input generation techniques use either (1) concrete execution or (2) symbolic execution that builds ...
CUTE: a concolic unit testing engine for C
In unit testing, a program is decomposed into units which are collections of functions. A part of unit can be tested by generating inputs for a single entry function. The entry function may contain pointer arguments, in which case the inputs to the unit ...
Fault detection effectiveness of source test case generation strategies for metamorphic testing
MET '18: Proceedings of the 3rd International Workshop on Metamorphic TestingMetamorphic testing is a well known approach to tackle the oracle problem in software testing. This technique requires the use of source test cases that serve as seeds for the generation of follow-up test cases. Systematic design of test cases is ...
Comments