Ken Koster
ABSTRACT
We propose a new language-independent, structural test adequacy criterion called state coverage. State coverage measures whether unit-level tests check the outputs and side effects of a program.
State coverage differs in several respects from existing test adequacy criteria, such as code coverage and mutation adequacy. Unlike other coverage-based criteria, state coverage measures the extent of checks of program behavior. And unlike existing fault-based criteria such as mutation adequacy, state coverage has been designed to be readily automated and to present users with easily understood test inadequacy reports.
An experiment showed strong positive correlations between the number of behavior checks and both state coverage and mutation adequacy.
- D. Astels. One assertion per test. http://www.artima.com/weblogs/viewpost.jsp?thread=35578.Google Scholar
- K. Beck. Kent Beck's Guide to Better Smalltalk. Cambridge University Press, 1998. Google ScholarDigital Library
- T. Burns. Effective unit testing. ACM Ubiquity, 1(42), 2001. Google ScholarDigital Library
- R. A. DeMillo, R. J. Lipton, and F. G. Sayward. Hints on test data selection: Help for the practicing programmer. IEEE Computer, 11(4), April 1978. Google ScholarDigital Library
- E. Duesterwald, R. Gupta, and M. L. Soffa. Rigorous data flow testing through output influences. In Second Irvine Software Symposium, 1992.Google Scholar
- R. G. Hamlet. Testing programs with the aid of a compiler. IEEE Trans. Softw. Eng., 3(4), July 1977. Google ScholarDigital Library
- C. A. R. Hoare. An axiomatic basis for computer programming. Commun. ACM, 12(10):576--580, 1969. Google ScholarDigital Library
- D. Hyland-Wood and D. Carrington. 2006 software engineering practices survey summary of results. Technical report, The University of Queensland, 2007. http://www.itee.uq.edu.au/~dwood/2006SEPSurvey/2006SEPSurveyResults.html.Google Scholar
- Y.-S. Ma, J. Offutt, and Y. R. Kwon. MuJava: An automated class mutation system. Software Testing, Verification and Reliability, 15(2):97--133, June 2005. Google ScholarDigital Library
- V. P. Ranganath and J. Hatcliff. Slicing concurrent Java programs using Indus and Kaveri, 2005. http://people.cis.ksu.edu/~rvprasad/publications/sttt05-submission.pdfGoogle Scholar
- S. Rapps and E. Weyuker. Selecting software test data using data flow information. IEEE Trans. Softw. Eng., 11(4):367--375, 1985. Google ScholarDigital Library
- P. Runeson. A survey of unit testing practices. IEEE Software, 23(4):22--29, 2006. Google ScholarDigital Library
- F. Tip. A survey of program slicing techniques. Journal of Programming Languages, 3:121--189, 1995.Google Scholar
- R. Torkar and S. Mankefors. A survey on testing and reuse. In Proceedings of the 2003 IEEE International Conference on Software-Science, Technology & Engineering, 2003. Google ScholarDigital Library
- M. Weiser. Program slicing. In Proceedings of the 5th International Conference on Software Engineering, 1981. Google ScholarDigital Library
- H. Zhu, P. A. V. Hall, and J. H. R. May. Software unit test coverage and adequacy. ACM Computing Surveys, 29(4):366--427, 1997. Google ScholarDigital Library
Index Terms
- State coverage: a structural test adequacy criterion for behavior checking
Recommendations
State coverage: a structural test adequacy criterion for behavior checking
ESEC-FSE '07: Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineeringWe propose a new language-independent, structural test adequacy criterion called state coverage. State coverage measures whether unit-level tests check the outputs and sideeffects of a program.
State coverage differs in several respects from existing ...
A state coverage tool for JUnit
ICSE Companion '08: Companion of the 30th international conference on Software engineeringWe present a JUnit test runner that informs users of missing behavior checks in their tests. The tool tracks variable updates and definitions over the course of a test execution and determines which variables influence which assertions via dynamic taint ...
Achieving scalable mutation-based generation of whole test suites
Without complete formal specification, automatically generated software tests need to be manually checked in order to detect faults. This makes it desirable to produce the strongest possible test set while keeping the number of tests as small as ...
Comments