An experimental comparison of ensemble of classifiers for bankruptcy prediction and credit scoring

Abstract

In this paper, we investigate the performance of several systems based on ensemble of classifiers for bankruptcy prediction and credit scoring.

The obtained results are very encouraging, our results improved the performance obtained using the stand-alone classifiers. We show that the method “Random Subspace” outperforms the other ensemble methods tested in this paper. Moreover, the best stand-alone method is the multi-layer perceptron neural net, while the best method tested in this work is the Random Subspace of Levenberg–Marquardt neural net.

In this work, three financial datasets are chosen for the experiments: Australian credit, German credit, and Japanese credit.

Introduction

An important issue in financial decision-making is to predict, timely and correctly, business failure (Atiya, 2001, Zhang et al., 1999) (e.g. bankruptcy prediction and credit scoring). The credit scoring models permit to discriminate between good credit group and bad credit group (Chen & Huang, 2003). The benefits obtained developing a reliable credit scoring system are (Tsai and Wu, 2007, West, 2000):

• reducing the cost of credit analysis;

• enabling faster decision;

• insuring credit collections and diminishing possible risk.

Several financial decision-making methods based on machine learning (examples of machine learning techniques used to solve the above financial decision-making problems are Atiya, 2001, Huang et al., 2004, Lee et al., 2006) use the multi-layer perceptron (MLP) (Haykin, 1999) as classifier. Other tested classifiers are the Decision Tree and the Support Vector machine. We want to stress that these studies show that the machine learning based systems are better than the traditional (statistical) methods for bankruptcy prediction and credit scoring problems (Huang et al., 2004, Ong et al., 2005, Vellido et al., 1999, Wong and Selvi, 1998). In Tsai and Wu (2007) the authors compare a single MLP classifier with multiple classifiers and diversified multiple classifiers on three datasets. However, they conclude that there is no an exact winner.

In Table 1, several machine learning based methods are compared.

The main drawbacks of the machine learning based methods proposed in the literature are (Tsai & Wu, 2007).

• In several works only one dataset is used to validate the proposed system.

• In several works only the accuracy is used to validate the proposed system (examples of exceptions are Lee et al., 2006, Lee et al., 2002, Tsai and Wu, 2007).

• Few papers study ensemble of classifiers (Frosyniotis et al., 2003, Ghosh, 2002, Kang and Doermann, 2003, Roli et al., 2004) on both credit scoring and bankruptcy prediction problems (Tsai and Wu, 2007, Wong and Selvi, 1998).

In this paper, we improve the results of Tsai and Wu (2007), we have made a deep study of the ensemble of classifiers for bankruptcy prediction and credit scoring. We have tested four different methods for creating an ensemble of classifiers and we have tested four different classifiers. We show that the Random Subspace method (RS) (Ho, 1998) improve the performance of the classifiers. In our test the best stand-alone classifier is MLP (as reported in several papers) but the best system is a Random Subspace of Levenberg–Marquardt neural nets.

The paper is organized as follows: in Section 2, a brief literature of the multiple classifiers is reported, in Section 3, the experimental results are presented. Finally, in Section 4, some concluding remarks are given.