An empirical study of regression testing techniques incorporating context and lifetime factors and improved cost-benefit models

Regression testing is an important but expensive activity, and a great deal of research on regression testing methodologies has been performed. In recent years, much of this research has emphasized empirical studies, including evaluations of the effectiveness and efficiency of regression testing techniques. To date, however, most studies have been limited in terms of their consideration of testing context and system lifetime, and have used cost-benefit models that omit important factors and render some types of comparisons between techniques impossible. These limitations can cause studies to improperly assess the costs and benefits of regression testing techniques in practical settings. In this paper, we provide improved cost-benefit models for use in assessing regression testing methodologies, that incorporate context and lifetime factors not considered in prior studies, and we use these models to compare several common methodologies. Our results show that the factors we consider (in particular, time constraints and incremental resource availability) can affect assessments of the relative benefits of regression testing techniques, and suggest that particular classes of techniques may compare differently across different types of test suites.