Abstract
Ensemble selection copes with the reduction of an ensemble of the predictive models to reduce its response time and increase its accuracy. A number of selection methods via greedy search of the space of all possible ensemble subsets have been recently proposed. The major issue of these algorithms is to construct an effective metric to supervise the search process. In this paper, we view the issue of ensemble problem from a new viewpoint: energy-based learning, and then contribute a novel metric called EBM (Energy-based Metric) to guide the search. Also, this metric takes into account the strength of the decision of the current ensemble. Empirical results show that using the proposed metric to select subensemble leads to significantly better accuracy results compared to state-of-the-art metrics.
Supported by the National Science Foundation of China (No. 60901078).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. John Wiley and Sons (2004)
Breiman, L.: Bagging predictors. Machine Learning, 123–140 (1996)
Freund, Y., Schapire, R.F.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55, 119–139 (1997)
Breiman, L.: Random forests. Machine learning 45, 5–32 (2001)
Zhang, D., Chen, S., Zhou, Z., Yang, Q.: Constraint projections for ensemble learning. In: 23rd AAAI Conference on Artificial Intelligence (AAAI 2008), pp. 758–763 (2008)
Zhang, Y., Burer, S., Street, W.N.: Ensemble pruning via semi-definite programming. The Journal of Machine Learning Research, 1315–1338 (2006)
Chen, H., Tino, P., Yao, X.: Predictive ensemble pruning by expectation propagation. IEEE Transactions on Knowledge and Data Engineering, 999–1013 (2009)
Guo, H., Fan, M.: Ensemble Pruning via Base-Classifier Replacement. In: Wang, H., Li, S., Oyama, S., Hu, X., Qian, T. (eds.) WAIM 2011. LNCS, vol. 6897, pp. 505–516. Springer, Heidelberg (2011)
Tamon, C., Xiang, J.: On the Boosting Pruning Problem. In: Lopez de Mantaras, R., Plaza, E. (eds.) ECML 2000. LNCS (LNAI), vol. 1810, pp. 404–412. Springer, Heidelberg (2000)
Martinez-Muverbnoz, G., Suarez, A.: Aggregation ordering in bagging. In: International Conference on Artificial Intelligence and Applications (IASTED), pp. 258–263. Acta Press, Calgary (2004)
Banfield, R.E., Hall, L.O., Bowyer, K.W., Kegelmeyer, W.P.: Ensemble diversity measures and their application to thinning. Information Fusion, 49–62 (2005)
Partalas, I., Tsoumakas, G., Vlahavas, I.P.: Focused Ensemble Selection: A Diversity-Based Method for Greedy Ensemble Selection. In: 18th European Conference on Artificial Intelligence, pp. 117–121 (2008)
MartÃnez-Muñoz, G., Hernández-Lobato, D., Suárez, A.: An analysis of ensemble pruning techniques based on ordered aggregation. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 245–259 (2009)
LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M.A., Huang, F.J.: A Tutorial on Energy-Based Learning. In: Predicting Structured Data. MIT Press (2006)
Fan, W., Chu, F., Wang, H., Yu, P.S.: Pruning and dynamic scheduling of cost-sensitive ensembles. In: Eighteenth National Conference on Artificial Intelligence. American Association for Artificial Intelligence, pp. 146–151 (2002)
Partalas, I., Tsoumakas, G., Vlahavas, I.P.: An ensemble uncertainty aware measure for directed hill climbing ensemble pruning. Machine Learning, 257–282 (2010)
Asuncion, D.N.A.: UCI machine learning repository (2007)
Quinlan, J.R.: C4.5: programs for machine learning. Morgan Kaufmann (1993)
Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhi, W., Guo, H., Fan, M. (2012). Energy-Based Metric for Ensemble Selection. In: Sheng, Q.Z., Wang, G., Jensen, C.S., Xu, G. (eds) Web Technologies and Applications. APWeb 2012. Lecture Notes in Computer Science, vol 7235. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29253-8_26
Download citation
DOI: https://doi.org/10.1007/978-3-642-29253-8_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-29252-1
Online ISBN: 978-3-642-29253-8
eBook Packages: Computer ScienceComputer Science (R0)