Abstract
We have recently proposed a novel algorithm for ensemble creation called GEMS (Genetic Ensemble Member Selection). GEMS first trains a fixed number of neural networks (here twenty) and then uses genetic programming to combine these networks into an ensemble. The use of genetic programming makes it possible for GEMS to not only consider ensembles of different sizes, but also to use ensembles as intermediate building blocks. In this paper, which is the first extensive study of GEMS, the representation language is extended to include tests partitioning the data, further increasing flexibility. In addition, several micro techniques are applied to reduce overfitting, which appears to be the main problem for this powerful algorithm. The experiments show that GEMS, when evaluated on 15 publicly available data sets, obtains very high accuracy, clearly outperforming both straightforward ensemble designs and standard decision tree algorithms.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Hansen, L.K., Salamon, P.: Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence 12(10), 993–1001 (1990)
Krogh, A., Vedelsby, J.: Neural network ensembles, cross validation, and active learning. In: Advances in Neural Information Processing Systems, vol. 2, pp. 650–659. Morgan Kaufmann, San Mateo (1995)
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Shapire, R.: The strength of weak learnability. Machine Learning 5(2), 197–227 (1990)
Zhou, Z.-H., Wu, J.-X., Jiang, Y., Chen, S.-F.: Genetic algorithm based selective neural network ensemble. In: 17th International Joint Conference of Artificial Intelligence, Seattle, WA, vol. 2, pp. 797–802 (2001)
Zhou, Z.-H., Wu, J.-X., Tang, W.: Ensembling Neural Networks: Many Could Be Better Than All. Artificial Intelligence 137(1-2), 239–263 (2002)
Opitz, D., Shavlik, J.: Actively searching for an effective neural-network ensemble. Connection Science 8(3/4), 337–353 (1996)
Johansson, U., Löfström, T., König, R., Niklasson, L.: Introducing GEMS - a Novel Technique for Ensemble Creation. In: 19th Florida Artificial Intelligence Research Society Conference (FLAIRS) 2006, AAAI Press, Melbourne (to appear, 2006)
Johansson, U., Löfström, T., Niklasson, L.: Obtaining Accurate Neural Network Ensembles. In: International Conference on Computational Intelligence for Modelling Control and Automation - CIMCA’2005 (in Press)
Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases, University of California, Department of Information and Computer Science (1998)
Quinlan, J.R.: See5 version 2.02 (2005), http://www.rulequest.com
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees, Wadsworth International Group (1984)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Johansson, U., Löfström, T., König, R., Niklasson, L. (2006). Genetically Evolved Trees Representing Ensembles. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Żurada, J.M. (eds) Artificial Intelligence and Soft Computing – ICAISC 2006. ICAISC 2006. Lecture Notes in Computer Science(), vol 4029. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11785231_64
Download citation
DOI: https://doi.org/10.1007/11785231_64
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35748-3
Online ISBN: 978-3-540-35750-6
eBook Packages: Computer ScienceComputer Science (R0)