Abstract
The literature suggests that an ensemble of classifiers outperforms a single classifier across a range of classification problems. This chapter provides a brief background on issues related ensemble construction and data set imbalance. It describes the application of ensembles of neural network classifiers and rule based classifiers to the prediction of potential defaults for a set of personal loan accounts drawn from a medium sized Australian financial institution. The imbalanced nature of the data sets necessitated the implementation of strategies to avoid under learning of the minority class and two such approaches (minority over-sampling and majority under-sampling) were adopted here. The ensembles outperformed the single classifiers, irrespective of the strategy that was used. The results suggest that an ensemble approach has the potential to provide a high rate of classification accuracy for problem domains of this type.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Baesens, B., Van Gestel, T., Stepanova, M., Vanthienen, J.: Neural Network Survival Analysis for Personal Loan Data. In: Proceedings of the Eighth Conference on Credit Scoring and Credit Control (CSCC VIII 2003), Edinburgh, Scotland (2003)
Bernardini, F.C., Monard, M.C., Prati, R.C.: Constructing Ensembles of Symbolic Classifiers. In: Proceedings of the 5th International Conference on Hybrid Intelligent Systems (HIS 2005), Rio de Janeiro Brazil, pp. 315–322. IEEE Press, Los Alamitos (2005)
Chawla, N.V., Hall, L.O., Bowyer, K., Kegelmeyer, W.P.: Learning Ensembles from Bites: A Scalable and Accurate Approach. Journal of Machine Learning 5, 421–451 (2004)
Cohen, G., Hilario, M., Sax, H., Hugonnet, S., Geissbuhler, A.: Learning form imbalanced data in surveillance of nosocomial infection. Artificial Intelligence in Medicine 37, 7–18 (2006)
Desai, V.S., Crook, J.N., Overstreet, G.A.: A Comparison of Neural Networks and Linear Scoring Models in the credit union environment. European Journal of Operational Research 95, 24–39 (1995)
Dietterich, T.G.: Ensemble Methods in Machine Learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857. Springer, Heidelberg (2000)
Drummond, C., Holte, R.C.: Severe Class Imbalance: Why Better Algorithms Aren’t the Answer. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 539–546. Springer, Heidelberg (2005)
Fahlman, S.E.: Faster-learning variations on back-propagation: An empirical study. In: Sejnowski, T.J., Hinton, G.E., Touretzky, D.S. (eds.) 1988 Connectionist Models Summer School. Morgan Kaufmann, San Mateo (1988)
Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Saitta, L. (ed.) Proceedings of the Thirteenth International Conference on Machine Learning, Bari, Italy, pp. 148–156. Morgan Kaufmann, San Francisco (1996)
Grannitto, P.M., Verdes, P.F., Ceccatto, H.A.: Neural network ensembles: evaluation of ag-gregation algorithms. Artificial Intelligence 163, 139–162 (2005)
Guo, H., Viktor, H.L.: Learning form Imbalanced Data Sets with Boosting and Data Gen-eration: The DataBoost-IM Approach. ACM SIGKDD Explorations Newsletter: Special Issue on Learning from Imbalanced Datasets 6, 30–39 (2004)
Hagan, M.T., Demuth, H.B., Beale, M.: Neural Network Design. PWS Publishing, Boston (1996)
Hansen, L.K., Salamon, P.: Neural Network Ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 993–1001 (1990)
Hastie, T., Tibshirani, R., Friedman, F.: The Elements of Statistical Learning: Data Mining, Inference and Prediction, Springer Series in Statistics, Springer, New York (2003)
Hoffman, F., Baesens, B., Martens, J., Put, F., Vanthienen, J.: Comparing a Genetic Fuzzy and a NeuroFuzzy Classifier for Credit Scoring. International Journal of Intelligent Systems 17, 1067–1083 (2002)
Huysmans, J., Baesens, B., Vanthienen, J., Van Getsel, T.: Failure Prediction with Self Or-ganising Maps. Expert Systems with Applications 30, 479–487 (2006)
Karray, F.O., de Silva, C.: Soft Computing and Intelligent Systems Design, Harlow, England. Pearson Education Limited, London (2004)
Ko, A.H.-R., Sabourin, R., deS (Jr.), B.A.: Combining Diversity and Classification Accuracy for Ensemble Selection in Random Subspaces. In: Proceedings 2006 International Joint Conference on Neural Networks, Vancouver Canada, pp. 2144–2151 (2006)
Kucheva, L.I.: Error Bounds for Aggressive and Conservative Adaboost. In: Windeatt, T., Roli, F. (eds.) MCS 2003. LNCS, vol. 2709, pp. 25–34. Springer, Heidelberg (2003)
Kucheva, L.I., Whitaker, C.J.: Measures of Diversity in Classifiers Ensembles and Their Relationship with the Ensemble Accuracy. Machine Learning 51, 181–207 (2003)
Lewis, E.M.: An Introduction to Credit Scoring. The Athena Press, San Rafael, California (1992)
Mays, E.: Handbook of Credit Scoring. Glenlake Publishing, Chicago (2001)
McNelis, P.D.: Neural Networks in Finance: Gaining Predictive Edge in the Market. Elsevier Academic Press, Burlington, MA (2005)
Nauck, D., Kruse, R.: NEFCLASS - A Neuro-Fuzzy Approach for the Classification of Data. In: George, K.M., Carrol, J.H., Deaton, E., Oppenheim, D., Hightower, J. (eds.) Proceedings of the 1995 ACM Symposium on Applied Computing, Nashville, Tennessee. ACM Press New York, New York (1995)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)
Scheurmann, E., Matthews, C.: Neural Network Classifiers in Arrears Management. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 325–330. Springer, Heidelberg (2005)
Srivastra, R.P.: Automating judgemental decisions using neural networks: a model for processing business loan applications. In: Agrawal, J.P., Kumar, V., Wallentine (eds.) Proceed-ings of the 1992 ACM Conference on Communications, Kansas City, Missouri ACM Press, New York (1992)
Torres-Sospedra, J., Fernandez-Redono, M., Hernandez-Espinosa, C.: Combination Methods for Ensembles of MF. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 131–138. Springer, Heidelberg (2005)
Vellido, A., Lisboa, P.J.G., Vaughan, J.: Neural Networks in Business: a survey of applications (1992-1998). Expert Systems with Applications 17, 51–70 (1999)
Wanas, N.M., Kamel, M.S.: Decision Fusion in Neural Network Ensembles. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2001), vol. 4, pp. 2952–2957. IEEE Press, Los Alamitos (2001)
West, D.: Neural network credit scoring models. Computers & Operations Research 27, 1131–1152 (2000)
Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)
Wu, Y., Arribas, J.I.: Fusing Output Information in Neural Networks: Ensemble Performa Better. In: Proceedings of the 25th Annual Conference of IEEE EMBS, Cancum, Mexico (2003)
Yule, G.U., Kendall, M.G.: An Introduction to the Theory of Statistics, 14th edn., Griffin, London (1950)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Matthews, C., Scheurmann, E. (2008). Ensembles of Classifiers in Arrears Management. In: Prasad, B. (eds) Soft Computing Applications in Business. Studies in Fuzziness and Soft Computing, vol 230. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-79005-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-540-79005-1_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-79004-4
Online ISBN: 978-3-540-79005-1
eBook Packages: EngineeringEngineering (R0)