A novel multi-stage hybrid model with enhanced multi-population niche genetic algorithm: An application in credit scoring

Highlights

• A novel multi-stage hybrid model is proposed and applied to credit scoring.

• Multi-population niche GA (MPNGA) is proposed to improve search efficiency.

• Feature/classifier selection enables the acquisition of optimal subset.

• The stacking-based ensemble is constructed to enhance predictive effectiveness.

• The proposed model is validated on five datasets over four performance metrics.

Abstract

In recent years, artificial intelligence and machine learning technology have made great progress and development. Various novel models have been constructed to enhance prediction performance of binary classification from different aspects. Credit scoring model is a typical application of artificial intelligence and machine learning technology. In this study, we propose a novel multi-stage hybrid model, which combines feature selection and classifier selection to obtain optimal feature subset and optimal classifier subset, then uses classifier ensemble to improve the prediction performance based on the two optimal subsets mentioned above. We also extend genetic algorithm, i.e., propose an enhanced multi-population niche genetic algorithm (EMPNGA), to improve the ability of optimization effectively by enhancing the selection, crossover, and mutation steps, and adding niche and migration steps. Furthermore, EMPNGA is applied to combine several filter methods and priori knowledge in feature selection and classifier selection respectively to further increase the search efficiency. The proposed model is applied to credit scoring to verify its prediction performance. Finally, five datasets and four evaluation metrics are applied in the experiment. The experimental results confirm that the performance of proposed model is superior to the other comparative models, proving that this study is of significance and effectiveness.

Introduction

In recent years, artificial intelligence and machine learning technology have been greatly developed. In previous studies, several typical classification models have been applied in binary classification, such as linear discriminant analysis (LDA; Fisher, 1936), logistic regression (LR; Hand & Kelly, 2002), decision tree (DT; Li, Ying, Tuo, & Li, 2004), support vector machine (SVM; Huang, Chen, Hsu, Chen, & Wu, 2004), and multilayer perceptron network (MLP; West, 2000).

In general, datasets for machine learning are typically multidimensional. However, irrelevant and redundant features not only reduce the prediction performance of a classification model but can also increase the computational complexity. Feature selection methods are recognized as promising approaches in machine learning, and it is applied to identify the key features to reduce the computing time cost of the classification models and improve the prediction performance. Some previous studies have explored feature selection methods, including Chen and Li, 2010, Hajek and Michalak, 2013, Maldonado, Pérez and Bravo, 2017, Oreski and Oreski, 2014, and Wang, Zhang, Bai, Mao (2017). But, there still remain new capabilities to be discovered and explored.

Ensemble models have also been widely considered to improve the performance of classification models in recent years. Many ensemble models have been applied to machine learning, such as homogeneous ensemble models based on DT, random forest (RF; Friedman, 2001), gradient boosting decision tree (GBDT; Friedman, 2001), and XGBoost (Chen & Guestrin, 2016). The heterogeneous ensemble models, which combine multiple base classifiers, have also garnered widespread attention (Ala’ Raj & Abbod, 2016a, and Ala'Raj and Abbod, 2016b, Xia, Liu, Da and Xie, 2018). Lessmann, Baesens, Seow, and Thomas (2015) proved that the performance of heterogeneous ensembles is frequently superior to individual classifiers. However, how to determine the most effective ensemble model for different datasets has not yet been completely solved. In addition, the problem complexity and computational time of classifier selection in the original feature is usually large. Therefore, effective classifier selection methods should be considered to obtain a more appropriate ensemble model within a certain complexity.

Credit scoring has gained considerable attention in financial industry owing to its importance in credit risk management. A small improvement in credit scoring model can bring large profits to financial institutions, therefore, many artificial intelligence and machine learning models have been applied to credit scoring to verify their performance in binary classification. In this study, we propose a novel multi-stage hybrid model, which combines feature selection and classifier selection, to obtain a superior prediction performance. Furthermore, an enhanced multi-population niche genetic algorithm (EMPNGA) is proposed to combine several filter methods and priori knowledge in feature selection and classifier selection respectively, to enable the acquisition of optimal feature/classifier subset. Then classifier ensemble is used to improve the prediction performance of the model based on these optimal subsets mentioned above. The proposed model is applied to credit scoring to verify its prediction performance in binary classification. The experimental results demonstrate that these multiple stages of the hybrid model have played a significant role in improving the prediction performance and the final prediction performance of the proposed model is superior to other comparative models. This confirms that the proposed model is effective and practical, and provides a new research direction for future machine learning research.

The remainder of this study is organized as follows. Section 2 describes related work regarding genetic algorithm, feature selection and classifier ensemble. Section 3 describes the mechanism of the proposed model. Section 4 presents the experimental design. Section 5 describes the experimental results and comparative analysis. The conclusions and future works are listed in Section 6.