Abstract
After deploying a classifier in production it is essential to support its lifecycle. This paper describes the application of an ensemble of classifiers to support two stages of the lifecycle of an on-line classifier used to underwrite life insurance applications: the monitoring of its decisions quality and the updating of the production classifier over time. All combinations of five classification methods and seven fusion methods were assessed from the perspective of accuracy and pairwise diversity of the classifiers, and accuracy, precision, and coverage of the fused classifiers. The proposed architecture consists of three off-line classifiers and a fusion module.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aggour, K., Pavese, M., Bonissone, P.: SOFT-CBR: A Self-Optimizing Fuzzy Tool for Case-Based Reasoning. In: Ashley, K.D., Bridge, D.G. (eds.) ICCBR 2003. LNCS, vol. 2689, pp. 5–19. Springer, Heidelberg (2003)
Bonissone, P.: The life cycle of a Fuzzy Knowledge-based Classifier. In: North American Fuzzy Information Processing Society (NAFIPS 2003), Chicago, IL, August 2003, pp. 488–494 (2003)
Bonissone, P.: Automating the Quality Assurance of an On-line Knowledge-Based Classifier By Fusing Multiple Off-line Classifiers. In: Proc. IPMU 2004, Perugia, Italy, pp. 309–316 (2004)
Bonissone, P., Cheetham, W.: Fuzzy Case-based Reasoning for Decision Making. In: IEEE Int. Conf. on Fuzzy Systems, Melbourne, Australia, pp. 995–998 (2001)
Bonissone, P., Goebel, K., Yan, W.: Classifier Fusion using Triangular Norms. In: Roli, F., Kittler, J., Windeatt, T. (eds.) MCS 2004. LNCS, vol. 3077, pp. 154–163. Springer, Heidelberg (2004)
Bonissone, P., Subbu, R., Aggour, K.: Evolutionary Optimization of Fuzzy Decision Systems for Automated Insurance Underwriting. In: IEEE Int. Conf. on Fuzzy Systems (FUZZ-IEEE 2002), Honolulu, Hawaii, USA, pp. 1003–1008 (2002)
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)
Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees, Wadsworth, Belmont, CA (1984)
Friedman, J.: Multivariate Adaptive Regression Splines. Annals of Statistics 19, 1–141 (1991)
Ho, T., Hull, J., Srihari, S.: Decision Combination in Multiple Classifier Systems. IEEE Trans. on Pattern Analysis and Machine Intelligence 16(1), 66–75 (1994)
Huang, Y., Suen, C.: A method of combining multiple experts for the recognition of unconstrained handwritten numerals. In: Trans. IEEE Pattern Analysis and Machine Intelligence, vol. 17(1), pp. 90–94 (1995)
Kuncheva, L.: Switching between selection and fusion in combining classifiers: An experiment. IEEE Transactions on SMC, Part B 32(2), 146–156 (2002)
Kuncheva, L., Whitaker, C.: Ten measures of diversity in classifier ensembles: Limits for two classifiers. In: Proceedings of IEE Workshop on Intelligent Sensor Processing, Birmingham, February 2001, vol. 10/1-10/6 (2001)
Langley, P., Iba, W., Thomson, K.: An analysis of Bayesian classifiers. In: Proceeding of National Conference on Artificial Intelligence (AAAI 1992), pp. 223–228 (1992)
Niyogi, P., Pierrot, J.-B., Siohan, O.: On decorrelating classifiers and combining them. MIT AI Lab (September 2001)
Partridge, D., Yates, W.: Engineering multiversion neural-net systems. Neural Computation 8, 869–893 (1996)
Patterson, A., Bonissone, P., Pavese, M.: Six Sigma Quality Applied Throughout the Lifecycle of and Automated Decision System. Journal of Quality and Reliability International (2005) (to appear)
Petrakos, M., Kannelopoulos, I., Benediktsson, J., Pesaresi, M.: The Effect of Correlation on the Accuracy of the Combined Classifier in Decision Level Fusion. In: Proceedings of IEEE 2000 International Geo-science and Remote Sensing Symposium, vol. 6 (2000)
Roli, F., Giacinto, G., Vernazza, G.: Methods for Designing Multiple Classifier Systems. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, pp. 78–87. Springer, Heidelberg (2001)
Tumer, K., Ghosh, J.: Error correlation and error reduction in ensemble classifiers. Connection Science 8, 385–404 (1996)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bonissone, P., Eklund, N., Goebel, K. (2005). Using an Ensemble of Classifiers to Audit a Production Classifier. In: Oza, N.C., Polikar, R., Kittler, J., Roli, F. (eds) Multiple Classifier Systems. MCS 2005. Lecture Notes in Computer Science, vol 3541. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11494683_38
Download citation
DOI: https://doi.org/10.1007/11494683_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26306-7
Online ISBN: 978-3-540-31578-0
eBook Packages: Computer ScienceComputer Science (R0)