Costs and benefits of early defect detection: experiences from developing client server and host applications

Abstract

Defect detection activities are generally seen as expensive and time consuming. This article reports on actual experiences from large projects. Based on data, dependencies between the costs of early defect detection, late defect detection and defect fixing are analysed and some guidelines are derived for timing and resource allocation to ensure cost-effective defect detection. Experience reports show that the costs of defect fixing in software development grow dramatically when the defects are found late, i.e. when the defects have already influenced the next phase. This common experience is supported by all the projects, two of which are considered in this paper. However, metrics applied to the data show that it is not always an order of magnitude that lies between the costs of early and late fixing of defects. Rather, the average cost of defects found late decreases with introduction of early defect detection. This is due to the fact that early document reviews leave only minor defects, e.g. in the requirements or the design. Major and therefore expensive defects tend to be found in those early reviews. Further analysis of the data led to the use of metrics that proved very useful when planning and executing the required defect detection activities. For example the number of remarks per 100 lines was used to decide how efficient reviews of the design have been and whether more or less resources should be allocated to these reviews in a running project. As a conclusion, it is emphasized that a substantial amount of a project's budget and time will be saved by introducing systematic defect detection early in the project. It is suggested that defect detection activities should start as soon as the requirements documents are declared finished.