Cost-sensitive Dictionary Learning for Software Defect Prediction

Abstract

In recent years, software defect prediction has been recognized as a cost-sensitive learning problem. To deal with the unequal misclassification losses resulted by different classification errors, some cost-sensitive dictionary learning methods have been proposed recently. Generally speaking, these methods usually define the misclassification costs to measure the unequal losses and then propose to minimize the cost-sensitive reconstruction loss by embedding the cost information into the reconstruction function of dictionary learning. Although promising performance has been achieved, their cost-sensitive reconstruction functions are not well-designed. In addition, no sufficient attentions are paid to the coding coefficients which can also be helpful to reduce the reconstruction loss. To address these issues, this paper proposes a new cost-sensitive reconstruction loss function and introduces an additional cost-sensitive discrimination regularization for the coding coefficients. Both the two terms are jointly optimized in a unified cost-sensitive dictionary learning framework. By doing so, we can achieve the minimum reconstruction loss and thus obtain a more cost-sensitive dictionary for feature encoding of test data. In the experimental part, we have conducted extensive experiments on twenty-five software projects from four benchmark datasets of NASA, AEEEM, ReLink and Jureczko. The results, in comparison with ten state-of-the-art software defect prediction methods, demonstrate the effectiveness of learned cost-sensitive dictionary for software defect prediction.

Appendices

Appendix A

In this section, we discuss the convexity of $R(A^{(i)},C)$ with respect to $A^{(i)}$. Firstly, we define two constants including matrix $Z_i=I-\frac{1}{n_i}E_i^i$ where $E^i_i\in {\mathbf {R}}^{n_i\times n_i}$ with all entries being 1 and $I\in {\mathbf {R}}^{n_i\times n_i}$ is the identity matrix, and vector ${\mathbf {p}}_i=\frac{1}{n_i}{\mathbf {v}}_i$ where ${\mathbf {v}}_i=[1,\ldots ,1]^T\in {\mathbf {R}}^{n_i}$ with all entries being 1.

With these two constants, we can rewrite the $R(A^{(i)},C)$ in Eq. (7) as follows

$$\begin{aligned} R(A^{(i)},C)=g(i)||A^{(i)}Z_i||_F^2-\sum _{j=1}^c(C_{ij}+C_{ji})||A^{(i)}{\mathbf {p}}_i-{\mathbf {u}}^{(j)}||_2^2+\eta ||A^{(i)}||_F^2, \end{aligned}$$

(15)

where

$$\begin{aligned} \left\{ \begin{array}{l} ||A^{(i)}Z_i||_F^2=\sum _{{\mathbf {a}}_k^{(i)}\in A^{(i)}} ||{\mathbf {a}}_k^{(i)}-{\mathbf {u}}^{(i)}||_2^2\\ A^{(i)}{\mathbf {p}}_i={\mathbf {u}}^{(i)} \end{array} \right. \end{aligned}$$

(16)

According to the definition of convex function, the convexity of $R(A^{(i)},C)$ with respect to $A^{(i)}$ depends on whether its Hessian matrix $\nabla ^2R(A^{(i)},C)$ is positive definite or not [67]. Specifically, by taking the derivative of $R(A^{(i)},C)$ in Eq. (15) with respect to $A^{(i)}$, the Hessian matrix $\nabla ^2R(A^{(i)},C)$ is as follows

$$\begin{aligned} \nabla ^2R(A^{(i)},C)=2g(i)Z_iZ_i^T-2\sum _{j=1}^c(C_{ij}+C_{ji}){\mathbf {p}}_i{\mathbf {p}}_i^T+2\eta I. \end{aligned}$$

(17)

In order to prove the positive definite of matrix $\nabla ^2R(A^{(i)},C)$, we substitute the constants $Z_i$ and ${\mathbf {p}}_i$ in Eq. (17) with $I-\frac{1}{n_i}E_i^i$ and $\frac{1}{n_i}{\mathbf {v}}_i$ respectively, and thus obtain

$$\begin{aligned} \nabla ^2R(A^{(i)},C)=2\Big (g(i)+\eta \Big )I-2E_i^i\left( \frac{1}{n_i}g(i)+\frac{1}{n_i^2}\sum _{j=1}^c(C_{ij}+C_{ji})\right) . \end{aligned}$$

(18)

Obviously, if matrix $\nabla ^2R(A^{(i)},C)$ is positive definite, all of its eigenvalues should be greater than 0. Due to the fact that the maximal eigenvalue of matrix $E_i^i$ is $n_i$ [20], the positive definite of $\nabla ^2R(A^{(i)},C)$ can be proved if

$$\begin{aligned} \Big (g(i)+\eta \Big )-n_i\left( \frac{1}{n_i}g(i)+\frac{1}{n_i^2}\sum _{j=1}^c(C_{ij}+C_{ji})\right) >0. \end{aligned}$$

(19)

By simple derivation, we obtain the condition $\eta >\frac{1}{n_i}\sum _{j=1}^c(C_{ij}+C_{ji})$, which can guarantee the positive definite of matrix $\nabla ^2R(A^{(i)},C)$ and thus make the $R(A^{(i)},C)$ be convex with respect to ${A}^{(i)}$.

Appendix B

See Tables 11, 12, 13 and 14.

Table 11 Indicator values on NASA dataset

Table 12 Indicator values on AEEEM dataset

Table 13 Indicator values on ReLink dataset

Table 14 Indicator values on Jureczko dataset