Adaptive ridge regression system for software cost estimating on multi-collinear datasets

Abstract

Cost estimation is one of the most critical activities in software life cycle. In past decades, a number of techniques have been proposed for cost estimation. Linear regression is yet the most frequently applied method in the literature. However, a number of studies point out that linear regression is prone to low prediction accuracy. The low prediction accuracy is due to a number of reasons such as non-linearity and non-normality. One less addressed reason is the multi-collinearities which may lead to unstable regression coefficients. On the other hand, it has been reported that multi-collinearity spreads widely across the software engineering datasets. To tackle this problem and improve regression's accuracy, we propose a holistic problem-solving approach (named adaptive ridge regression system) integrating data transformation, multi-collinearity diagnosis, ridge regression technique and multi-objective optimization. The proposed system is tested on two real world datasets with the comparisons with OLS regression, stepwise regression and other machine learning methods. The results indicate that adaptive ridge regression system can significantly improve the performance of regressions on multi-collinear datasets and produce more explainable results than machine learning methods.

Introduction

The production of low cost and high quality software in a short time has become the ultimate goal of software industry. To achieve this goal, the software development processes need to be well managed and controlled. One of the most important activities is that of estimating the cost devoted to build the software. This task is known as Software Cost Estimation (Boehm, 1981). Aiming at accurate estimation, several techniques have been published in the past decades. Such as expert judgment (Jorgensen, 2004, Gruschke and Jorgensen, 2008), parametric models (Boehm, 1981, Albrecht and Gaffney, 1983, Puntnam and Myers, 1991), machine learning methods (Shepperd and Schofield, 1997, Heiat, 2002, Pendharkar et al., 2005, Kumar et al., 2008, Keung et al., 2008, Mendes and Mosley, 2008, Li et al., 2009a, Li et al., 2009b) and linear regression methods (Miyazaki et al., 1994, Costagliola et al., 2005, Berlin et al., 2009, Huang et al., 2008).

According to a recent overview conducted by Jorgensen and Shepperd (2007), the linear regression is yet the most frequently applied method in the cost estimation literature. A large number of studies used linear regression methods (especially the OLS regression) as a benchmark against their proposed methods and concluded that regressions have lower accuracies than the new methods. However, as Kitchenham and Mendes (2009) recently pointed out that, these conclusions might be biased due to the inappropriate use of regression methods. Some restrictions of linear regressions such as homoscedasticity, moderate outliers, and normal errors might be ignored by the users. Among all restrictions, independence of the explanatory variables (or project features) is a less addressed one in the cost estimation literature. This restriction is violated when the multi-collinearity phenomenon appears. Multi-collinearity means two or more explanatory variables in a multiple regression model are highly correlated in linear form and multi-collinearity often causes linear regression losing stability and effectiveness (Kutner et al., 2005).

On the other hand, the inter-correlated explanatory variables (or project features) widely spreads across the software engineering datasets (Shepperd and Kadoda, 2001, Mendes et al., 2003). For example, the ‘number of lines of source code’ is often regarded to be positively related to the ‘number of inputs’, ‘number of outputs’, etc. In literature, there are some techniques to alleviate the effects of multi-collinearity: such as getting additional or dropping some explanatory variables (Neter et al., 1983). However, it is either difficult to get appropriate variables out from large amount of project features such as the ISBSG provided (ISBSG, 2007), or time-consuming to gather the information needed to create a new variable. On the other hand, dropping some variables might lead to only a few explanatory variables in the regression equation. Too few variables will reduce the explainability of the regression equation and make it vulnerable to the changing collinearities among explanatory variables.

Ridge regression technique (Hoerl and Kennard, 1970) provides a good alternative to deal with multi-collinearity which makes full use of the existing data and avoids adding or dropping explanatory variables. It also holds the potential (via ridge parameter) to improve the fitting and prediction accuracy of the regression methods. As ridge regression implements the central idea of the regularization theory (Tikhonov and Arsenin, 1977), some studies (Agarwal et al., 2007, Yu and Liong, 2007) have pointed out that it can achieve equally good or even better prediction performances than some machine learning methods such as support vector machine and artificial neural networks. In addition, the ridge regression equation is more transparent than the black-box liked machine learning methods especially neural networks.

Ridge regression has been successfully applied in a number of research areas such as biology (Goeman, 2008), environmental science (Hessami et al., 2008), hydrology (Chokmani et al., 2008), and nuclear science (Zhou et al., 2001). More recently, ridge regression has been introduced for the estimation issues in software engineering. Nguyen et al. (2008) applied ridge regression technique to estimate the constrained coefficients for the COCOMO models. Papadopoulos et al. (2009) utilized ridge regression to generate confidence intervals for effort estimation. Parsa et al. (2008) applied ridge regression to produce classification scores to filter out the redundant features. However, none of the previous studies have been focused on using ridge regression to resolve the multi-collinearity problem in the software cost dataset.

Based on ridge regression, we propose a novel problem-solving approach, namely adaptive ridge regression (ARR) system, combining different techniques for cost estimation on multi-collinear datasets. The ARR system consists of data transformation, multi-collinearity diagnosis, ridge regression technique and a multi-objective optimization to train the ridge parameter. The rest of this paper is organized as follows: Section 2 introduces multi-collinearity diagnosis and ridge regression; Section 3 describes the proposed adaptive ridge regression (ARR) system; Section 4 presents the real world datasets and procedures for the empirical validation; Section 5 describes the experimental results and comparisons; Section 6 presents the threads to validity; the last section summarizes this work and points out possible future directions.