Abstract
Ensemble pruning is an important issue in the field of ensemble learning. Diversity is a key criterion to determine how the pruning process has been done and measure what result has been derived. However, there is few formal definitions of diversity yet. Hence, three important factors that should be further considered while designing a pruning criterion is presented, and then an effective definition of diversity is proposed. The experimental results have validated that the given pruning criterion could single out the subset of classifiers that show better performance in the process of hill-climbing search, compared with other definitions of diversity and other criteria.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Dietterich, T.G.: Ensemble Methods in Machine Learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000)
Shipp, C., Kuncheva, L.: Relationships between combination methods and measures of diversity in combing classifiers. Information Fusion 3(2), 135–148 (2002)
Kuncheva, L., Whitaker, C.: Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning 51(2), 181–207 (2003)
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Freund, Y., Schapire, R.: Experiments with a new boosting algorithm. In: Proceedings of the Thirteenth International Conference on Machine Learning (ICML 1996), pp. 148–156 (1996)
Ho, T.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)
Caruana, R., Niculescu-Miil, A., Crew, G., et al.: Ensemble Selection from Libraries of Models. In: Proceedings of the Twenty-First International Conference (ICML 2004), pp. 18–25 (2004)
Martinez-Munoz, G., Suarez, A.: Using boosting to prune bagging ensembles. Pattern Recognition Letters 28, 156–165 (2007)
Tsoumakas, G., Partalas, I., Vlahavas, I.: An Ensemble Pruning Primer. In: Okun, O., Valentini, G. (eds.) Applications of Supervised and Unsupervised Ensemble Methods. SCI, vol. 245, pp. 1–13. Springer, Heidelberg (2009)
Martinez-Munoz, G., Suarez, A.: Aggregation ordering in bagging. In: Proceedings of the 2004 International Conference on Artificial Intelligence and Applications, pp. 258–263 (2004)
Yang, Y., Korb, K.B., Ting, K.M., Webb, G.I.: Ensemble Selection for SuperParent-One-Dependence Estimators. In: Zhang, S., Jarvis, R.A. (eds.) AI 2005. LNCS (LNAI), vol. 3809, pp. 102–112. Springer, Heidelberg (2005)
Banfield, R., Hall, L., Bowyer, K., et al.: Ensemble diversity measures and their application to thinning. Information Fusion 6(1), 49–62 (2005)
Partalas, I., Tsoumakas, G., Vlahavas, I.: Focused ensemble selection: a diversity-Based method for greedy ensemble selection. In: Proceedings of the 18th European Conference on Artificial Intelligence (ECAI 2008), pp. 117–121 (2008)
Caruana, R., Niculescu-Mizil, A.: Data mining in metric space: an empirical analysis of supervised learning performance criteria. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2004), pp. 69–78 (2004)
Kapp, M., Sabourin, R., Maupin, P.: An empirical study on diversity measures and margin theory for ensembles of classifiers. Information Fusion (2007)
Skalak, D.: The sources of increased accuracy for two proposed boosting algorithms. In: Proceedings of the 11th AAAI Conference on Artificial Intelligence, pp. 120–125 (1996)
Giacinto, G., Roli, F.: Design of effective neural network ensembles for image classification processes. Image Vision and Computing Journal 19(9), 699–707 (2001)
Kohavi, R., Wolpert, D.: Bias plus variance decomposition for zero-one loss functions. In: Proceedings of the Thirteenth International Conference on Machine Learning, pp. 275–283 (1996)
Hashem, L., Salamon, P.: Neural network ensembles. IEEE Transaction on Pattern Analysis and Machine Intelligence 12(10), 993–1001 (1990)
Tang, E., Suganthan, P., Yao, X.: An analysis of diversity measures. Machine Learning 65(1), 241–271 (2006)
Schapire, R., Freund, Y., Bartlett, P., et al.: Boosting the margin: a new explanation for the effectiveness of voting methods. In: Proceedings of the Fourteenth International Conference on Machine Learning (ICML 1997), pp. 322–330 (1997)
Witten, I., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations, pp. 365–368. Morgan Kaufmann Publishers, San Francisco (2000)
Asuncion A, Newman, D.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine, CA (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fu, B., Wang, Z., Pan, R., Xu, G., Dolog, P. (2013). An Integrated Pruning Criterion for Ensemble Learning Based on Classification Accuracy and Diversity. In: Uden, L., Herrera, F., Bajo Pérez, J., Corchado RodrÃguez, J. (eds) 7th International Conference on Knowledge Management in Organizations: Service and Cloud Computing. Advances in Intelligent Systems and Computing, vol 172. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30867-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-30867-3_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-30866-6
Online ISBN: 978-3-642-30867-3
eBook Packages: EngineeringEngineering (R0)