DBOS_US: a density-based graph under-sampling method to handle class imbalance and class overlap issues in software fault prediction

Abstract

Improving software quality by predicting faults during the early stages of software development is a primary goal of software fault prediction (SFP). Various machine learning models help to predict software faults. However, the imbalanced class distribution in the datasets may challenge some traditional learning approaches as they are more biased toward the majority class. The existence of class overlapping makes the prediction difficult owing to learning the minority class inaccurately. In addition to that, high data dimensionality makes the classification process complex and time-consuming. To enhance the performance of the classifier, handling these data quality issues is a big concern. This paper proposes a hybrid density-based method DBOS_US to address the class imbalance, noise, and class overlap in SFP. Initially, the density-based overlap removal (DBO) clustering algorithm is proposed to filter noisy and overlapped instances. Then, a graph-based algorithm, ShapeGraph, is adapted to handle imbalanced classes. The objective of the proposed method DBOS_US is to improve the performance of the traditional SFP classifiers. The experiments are conducted on 11 benchmark datasets from the PROMISE repository using six machine learning models (SVM, DT, KNN, NB, RF, and boosting). The experimental findings and statistical analysis revealed that the proposed method outperforms seven state-of-the-art techniques in terms of Area Under the Curve (AUC), G-mean, Recall (PD), and Probability of False alarms (PF). The proposed method improves the average values of G-mean, Recall, and AUC by at least 2.5%, 8.8%, and 1.2%, respectively.

Appendices

Appendix

This section divides into three parts- A1 describes the description of FLDA. A2 describes the results of Baseline, DBOS_US and its constituents' classification results in terms of AUC, G-mean, PF and Recall in the form of Tables 4, 5, 6 and 7, respectively. A3 describes the comparison of the average proposed methods' performance corresponding to each dataset with other state-of-the-art methods in terms of AUC, G-mean, PF and Recall in the form of Tables 8, 9, 10, 11

Appendix A1-FLDA

Consider that there are n training sample vectors represented by $\left\{ {t_{i} } \right\} _{i = 1}^{n}$, for m classes: C₁,C₂…C_m, and n_j samples belong to the jth class, i.e., $= \sum\nolimits_{j = 1}^{m} {n_{j} }$. Assume µ be the mean of all training samples, i.e., $\mu = (1/n)\sum\nolimits_{i = 1}^{n} {t_{i} }$, and µ_j be the mean of the jth class, i.e., $\mu_{j} = \left( {1/n_{j} } \right)\sum\nolimits_{{t_{i} \in C_{j} }}^{n} {t_{i} }$, Then, the within-class scatter matrix S_w, and the between-class scatter matrix S_B are stated, respectively, as

$$S_{{\text{w}}} = \mathop \sum \limits_{{t_{i} \in C_{j} }} \left( {t_{i} - \mu_{j} } \right) \left( {t_{i} - \mu_{j} } \right)^{T}$$

(10)

$$S_{{\text{B}}} = \mathop \sum \limits_{j = 1}^{m} n_{j} \left( {\mu_{j} - \mu } \right) \left( {\mu_{j} - \mu } \right)^{T}$$

(11)

The objective is to determine a transform vector v that maximizes the Raleigh quotient, which is stated as $q = \frac{{v^{T } S_{{\text{B }}} v}}{{v^{T } S_{{\text{w }}} v}}$ , where v can evaluate by $S_{{\text{B }}}$ v = λ $S_{{\text{w }}}$ v, and λ is a generalized eigenvalue. There are m-1 eigenvectors incorporated with m-1 nonzero eigenvalues owing to S_B rank is m-1. The m classes are predicted to be clearly distinguished in this low-dimensional space.

Algorithm 4

FLDA (Dimensionality reduction stage)

Appendix A2

See Tables 4, 5, 6 and 7.

Table 4 DBOS_US and its constituents' classification results in terms of AUC

Table 5 DBOS_US and its constituents' classification results in terms of G-mean

Table 6 DBOS_US and its constituents' classification results in terms of PF

Table 7 DBOS_US and its constituents' classification results in terms of Recall

Appendix A3

See Tables 8, 9, 10 and 11.

Table 8 Comparing the average proposed methods' performance corresponding to each dataset with other state-of-the-art methods in terms of AUC

Table 9 Comparing the average proposed methods' performance corresponding to each dataset with other state-of-the-art methods in terms of G-mean

Table 10 Comparing the average proposed methods' performance corresponding to each dataset with other state-of-the-art methods in terms of PF

Table 11 Comparing the average proposed methods' performance corresponding to each dataset with other state-of-the-art methods in terms of Recall