Partition testing with usage models
Section snippets
Statistical testing
Statistical testing is the comprehensive application of statistical science to the solution of software testing problems. The population of interest is all possible uses of the software under test, and this population must be represented in a form that facilitates posing and solving problems about the software and the testing process. The representation we use is the Markov chain usage model, which is described below. Further information about Markov chain usage models is presented in Ref. [1].
Partition testing
Sampling theory is a mature area of statistics. To take advantage of stratified sampling methods it is necessary to define the sample in terms of a mathematical partition of the population. Sample allocations are then calculated for the blocks of the partition.
Testing software by using random samples of test cases drawn from blocks of a partition defined on the population of all test cases is called “partition testing.” In this paper testing the use of a random sample drawn from the
Model-based partitioning strategies
A good partitioning strategy is one that results in blocks with different proportions of failures. The greatest gain in precision is realized when all blocks of the partition are homogeneous, i.e. all tests within a block either fail or all of the tests succeed [2], [3], [4], [5].
Given usage model M, let W be a matrix where Wi,j is a pre-test estimate of the probability of encountering a failure while executing the code to achieve transition from state-i to state-j, that is, while traversing
Conclusions
If tests are allocated to partition blocks in the optimal manner, partition random testing will always perform at least as well as simple random testing. Gains are realized from partition testing when the partitioning strategy creates near-homogeneous blocks, i.e. blocks in which all tests either fail or succeed. Therefore the choice of a partitioning scheme is very important.
The partitioning schemes presented in this paper use pre-test estimates of the failure rate of the software to create
References (9)
- et al.
Application of Statistical Science to Testing and Evaluating Software Intensive Systems
(1998) - et al.
Partition testing does not inspire confidence
IEEE Trans. Software Engng
(1990) - et al.
Analyzing partition testing strategies
IEEE Trans. Software Engng
(1991) Sampling Techniques
(1977)
Cited by (7)
Estimating confidence interval of software reliability with adaptive testing strategy
2014, Journal of Systems and SoftwareImpartial evaluation in software reliability practice
2005, Journal of Systems and SoftwareStopping criteria for statistical testing
2000, Information and Software TechnologyA mathematical modeling framework for software reliability testing
2007, International Journal of General SystemsSoftware partition testing based on execution profiles filtering
2006, Hunan Daxue Xuebao/Journal of Hunan University Natural SciencesUsing information about functions in selecting test cases
2005, Proceedings of the 1st International Workshop on Advances in Model-Based Testing, A-MOST '05