A hybrid imbalanced classification model based on data density

Abstract

Imbalanced data are widely available in the real world, and it is difficult to effectively identify the minority class instances in imbalanced data. Various imbalanced classification models have been proposed. However, these models neglect the data density and the location of instances which can be important factors affecting classification performance. To tackle this issue, this paper proposes a hybrid imbalanced classification model based on data density (HICD). In data-level, the density-based resampling method is presented. The data partition Algorithm is given, which divides the data space into five regions based on the data density. The corresponding subsets are generated by sampling from the divided regions to improve the recognition of different classes of instances. In algorithm-level, we construct the corresponding ensemble models for different classes of instances. In addition, the model selection Algorithm is presented. On this basis, an appropriate model is selected for each instance based on its distribution. The performance of the proposed HICD was evaluated on 18 imbalanced datasets in the real world in terms of recall, the area under the roc curve (AUC), and G-mean. The experimental results validate that our method has better performance than other competitive algorithms in imbalanced classification.

Introduction

Imbalanced data is a type of dataset that the number of instances is not uniform in the different classes. In an imbalanced dataset, a small number of instances exist in minority class, while a large number of instances exist in majority class. There are some challenges in training a model that can effectively identify the minority class instances in the imbalanced dataset, which is the imbalanced classification problem [1]. In imbalanced classification, the classifier will bias the majority class and neglect the minority class instances. Hence, this classifier will yield a low recall, and lower the AUC and G-mean. However, in the real world, the minority class instances always contain more crucial information, such as medical diagnosis [2], [3], [4], [5], fraud detection [6], [7], [8] or credit evaluation [9], [10], [11]. Only a few fraudulent instances and many normal instances are included in fraud detection. However, it is more critical to accurately identify fraudulent instances to prevent financial losses. As a result, it is especially vital to enhance the recall of minority class instances regarding imbalanced classification.

To address the pressing need for imbalanced classification, a series of solutions have been proposed [12], [13], [14], [15]. These methods consist of three classes. The first type resorts to the data-level method of sampling original datasets, which mainly includes over-sampling targeting the minority class, under-sampling targeting the majority class, and hybrid-sampling targeting the minority class and majority class [16], [17], [18]. Another common set of approaches is algorithm-level methods [19], [20], [21]. Besides, algorithm-level methods are free of changes to the data distribution and instead enhance the classification performance by improving the classifiers. Algorithm-level methods have cost-sensitive learning [22], ensemble learning [23], etc. The last category is hybrid methods, which combines data-level with algorithm-level methods [24], [25], [26]. In hybrid methods, data-level methods can alter the data distribution to reduce the degree of data imbalance, moreover, algorithm-level methods can change the learning process to improve the classification performance.

In contrast, hybrid methods have great advantages. Therefore, to improve the classification performance of imbalanced data, this paper proposes a hybrid imbalanced classification model based on data density. There are three main types of data-level methods: over-sampling, under-sampling, and hybrid-sampling. Under-sampling methods may lose crucial information when removing majority class instances. Over-sampling methods can lead to an overfitting problem. Hybrid-sampling methods can overcome these deficiencies. Since the class distribution of imbalanced data is not uniform, the data density and the location of instances may affect the effectiveness of the sampling algorithm. In this paper, the density-based resampling method is proposed. Based on data density, the data space is divided into different regions. Instances from different regions are sampled using the Bootstrap method to generate corresponding subsets for different classes of instances. To stabilize the classification performance, several weak classifiers can be combined to determine the output. This approach is ensemble learning capable of improving the classification accuracy and reducing the classification error [27]. In this paper, we construct corresponding ensemble models for different classes of instances. In addition, the model selection Algorithm is given. With the model selection algorithm, a suitable model is chosen for each instance based on its distribution.

To obtain a better binary imbalanced data classification model, this paper focuses on a hybrid imbalanced classification model based on data density. The main contributions are as follows:

• To improve the classification performance of imbalanced data, we propose HICD for the binary imbalanced data classification task.

• To find the characteristics of different classes of instances, we present the data partition algorithm, which divides the data space into five regions based on data density. Furthermore, by sampling from the divided regions, we propose the density-based resampling method.

• For different classes of instances, the corresponding ensemble models are constructed. In addition, we present the model selection algorithm. On this basis, an appropriate model is selected for each instance based on its distribution.

The rest of this paper is arranged as follows. Section 2 introduces recent related works. Section 3 describes the HICD in detail. Section 4 presents the experimental setting and compares HICD with other state-of-the-art methods on 18 datasets. Section 5 concludes the paper.