Combined weighted multi-objective optimizer for instance reduction in two-class imbalanced data problem

Abstract

Instance reduction from class-balanced data has been investigated in much research. However, there is a lack of studies on class-imbalanced data. Learning from imbalanced data lately has attracted a lot of attention due to the practical applications. In the case of two-class imbalanced data, the instances from one class, majority class, are more numerous than the instances from the other class, which is a minority class. The present paper aims to introduce a new instance reduction method that preserves between-class distributions in the balanced data and handles minority class instance reduction in two-class imbalanced data, efficiently. The proposed method solves the instance reduction issue from an unconstrained multi-objective optimization problem aspect. Accordingly, a new combined weighted optimizer is designed. By employing the chaotic krill herd evolutionary algorithm, both the minority and majority class spaces with the accelerated convergence are explored. Through this method, the original data set is purged of those instances that decrease accuracy, and Gmean. The performance has been evaluated on both imbalanced and balanced data sets collected from the UCI repository by the 10-fold cross-validation method. Evaluations show that the proposed method outperforms state-of-the-art methods in terms of classification accuracy, Gmean, reduction rates, and computational time.

Introduction

Instance reduction is one of the most important preprocessing steps in many machine learning tasks (Luan and Dong, 2018, Yang et al., 2019, Shakiba and Hooshmandasl, 2016). Because of the difficulties in handling the voluminous data, removing redundant, erroneous, or noisy instances is needed to perform before applying the data mining tasks such as instance-based learning methods, e.g., kNN and SVM. Instance reduction alleviates the high storage requirement and sensitivity to noise. Also, it decreases the complexity of computation needed to learn a high-quality classifier (Song and Chen, 2018, Yu et al., 2018). One of the most challenging areas in this field is handling between-class distributions (Wang and Mao, 2019). This means that the between-class distribution before and after data reduction should not be changed significantly. An inappropriate reduction method may eliminate more instances from one class, causing imbalanced data sets.

Instance reduction from class-balanced data has been investigated in much research. However, there is a lack of studies on class-imbalanced data. Recently learning from imbalanced data has attracted a lot of attention due to the practical applications in many domains, such as computer vision, network intrusion detection, medical diagnosis, and fraud detection (Kaur et al., 2019). The two-class imbalanced data deal with data containing instances from two classes and instances from one class, majority class, outnumber instances from the other class, that is minority class.

The problem of imbalanced data may cause some difficulties in learning tasks. Most of the studies employ traditional methods for learning from imbalanced data, but nevertheless, acceptable results may not be achieved because the traditional methods often give good coverage of the majority instances, but minority classes are neglected. Even if the obtained accuracy is significant, the result is not reliable because the cardinality of the minority class is very small compared to the cardinality of the majority class. Therefore, maintaining the between-class distribution is an important issue in the instance reduction problem. Although the minority class instances are typically essential in imbalanced data classification, they can be treated as outliers or noise. Accordingly, these instances should not be reduced when an instance reduction method is applied on imbalanced data sets. Hence, utilizing special methods is a necessity.

Many different approaches have been proposed (Díez-Pastor et al., 2015b, Díez-Pastor et al., 2015a, Dong et al., 2018, Fernández et al., 2018b, Fernández et al., 2018a) to deal with the instance reduction for imbalanced data. Among them, one can mention data-level methods that can be categorized into two main groups: undersampling in which the size of the majority class is decreased (Prasad et al., 2019), and oversampling in which the size of the minority class is increased (Krawczyk et al., 2019), cost-sensitive (Ling et al., 2006), and ensemble-based methods (Galar et al., 2011). The evolutionary-based methods with integration of the undersampling techniques have been gained attention. Some studies suggest that evolutionary-based methods outperform the non-evolutionary models in both instance reduction (de Haro-García et al., 2018, García-Pedrajas et al., 2014, Wang et al., 2015) and imbalanced data sets analysis (García and Herrera, 2009). Krill Herd Algorithm (KHA), as one of the most potent evolutionary algorithms, has been applied widely in recent years and has been showing acceptable results (Jensi and Jiji, 2016, Adhvaryyu et al., 2017, Niu et al., 2017). This evolutionary algorithm can effectively explore/exploit the solution spaces of different landscapes and dimensionality, and finally, converge to acceptable regions within the solution space (Mozaffari et al., 2017). However, its global exploration ability is not as strong as its local exploration ability, and it cannot always converge rapidly. The modification of the KHA based on the chaotic concept has been presented for tackling this problem.

In this paper, an evolutionary undersampling method for balanced and imbalanced data distributions is proposed. The proposed method generates a reduced set composed of those instances that enhance the performance of the classifiers. Also, it reduces the computational time needed to learn a classifier. The proposed method controls between-class distribution and protects minority class instances.

In the proposed method, instance reduction is viewed as an unconstrained multi-objective optimization problem. By using the chaotic krill herd evolutionary algorithm, both the minority and majority class spaces are explored with the accelerated convergence. The krill individuals (instances) are evaluated by a new combined weighted fitness function regarding contradicting criteria: accuracy, Geometry mean (Gmean), and reduction rate. Note that accuracy and Gmean contradict the reduction rate. In some cases, accuracy and consequently Gmean go against the reduction rate, i.e., accuracy and Gmean are getting better while the reduction rate is getting low or vice versa. Utilizing the designed decision surface, the krill individuals that have the best fitness are found. The output of the proposed method is a set that is purged of those instances that decrease accuracy and Gmean.

The rest of this paper is organized as follows. Section 2 reviews the related work. Section 3 elaborates on the proposed method, while the experimental results using benchmark data sets are presented in Section 4. A discussion on the conducted experiments is presented in Section 5. Finally, the conclusions and future work are drawn in Section 6.