Comprehension and Evolution of Combinatorial Models and Test Plans

Combinatorial test design (CTD) [1] is an effective test design technique, considered to be a testing best practice. CTD provides automatic test plan generation, but it requires a manual definition of the test space in the form of a combinatorial model, consisting of parameters, their respective values, and constraints on the value combinations. A valid test in the test space is defined to be an assignment of one value to each parameter that satisfies the constraints. A CTD algorithm automatically constructs a subset of the set of valid tests, termed a test plan, which covers all valid value combinations of every t parameters, where t is usually a user input. Such a test plan is said to achieve 100% t-way interaction coverage. A significant combinatorial reduction is achieved in the size of the resulting test plan (compared to manually designed test plans for example) because the tests generated by the CTD algorithm are very different from one another, maximizing their added value -- each of them covers as many unique t-way value tuples as possible. Note that tests produced by the algorithm are parameter-value assignments. Generating executable tests from them is often a separate, manual effort.