Improving the Predictive Power of AdaBoost: A Case Study in Classifying Borrowers

Abstract

Boosting is one of the recent major developments in classification methods. The technique works by creating different versions of a classifier using an adaptive resampling procedure and then combining these classifiers using weighted voting. In this paper, several modifications of the original version of boosting, the AdaBoost algorithm introduced by Y. Freund and R.E. Schapire in 1996, will be explained. These will be shown to substantially improve the predictive power of the original version. In the first modification, weighted error estimation in AdaBoost is replaced by unweighted error estimation and this is designed to reduce the impact of observations that possess large weight. In the second modification, only a selection of base classifiers, i.e. those that contribute significantly to predictive power of the boosting model, will be included in the final model. In addition to these two modifications, we will also utilise different classification techniques as base classifiers in order to product a final boosting model. Applying these proposed modifications to three data sets from the banking industry provides results which indicate a significant and substantial improvement in predictive power over the original AdaBoost algorithm.