Experimental designs for validating metrics and applying them across multiple projects

Abstract

We explain why it is important to validate metrics for use on multiple projects and how the risk of doing so can be assessed. We also point out that by prototyping the measurement plan on a project, we can eliminate the risk inherent in the process of validating metrics on one project and applying them on another project, where the two projects may have significant differences in application and development environment characteristics. In order to support the application of metrics across multiple projects, there must be reuse of methodologies, metrics, and metrics processes. A metrics methodology is reusable when there is a process associated with the methodology that can be applied across projects. Both the metrics and the process for applying metrics must be reusable. Metrics are reusable when validated metrics can be applied across projects, using the methodology. An example is given of assessing risk by using the confidence limits of a metric to evaluate the consequences of best-case and worst-case outcomes of using metrics on multiple projects.