Abstract
Self-poised ensemble learning is based on the idea of introducing an artificial innovation to the map to be predicted by each machine in the ensemble such that it compensates the error incurred by the previous one. We will show that this approach is equivalent to regularize the loss function used to train each machine with a penalty term which measures decorrelation with previous machines. Although the algorithm is competitive in practice, it is also observed that the innovations tend to generate an increasedly bad behavior of individual learners in time, damaging the ensemble performance. To avoid this, we propose to incorporate smoothing parameters which control the introduced level of innovation and can be characterized to avoid an explosive behavior of the algorithm. Our experimental results report the behavior of neural networks ensembles trained with the proposed algorithm in two real and well-known data sets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vedelsby, J., Krogh, A.: Neural network ensembles, cross-validation and active learning. Neural Information Processing Systems 7, 231–238 (1995)
Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Drucker, H.: Improving regressors using boosting techniques. In: Fourteenth International Conference on Machine Learning, pp. 107–115 (1997)
Harris, R., Brown, G., Wyatt, J., Yao, X.: Diversity creation methods: A survey and categorisation. Information Fusion Journal 6(1), 5–20 (2004)
Kuncheva, L.I., Kountchev, R.K.: Generating classifier outputs of fixed accuracy and diversity. Pattern Recognition Letters (23), 593–600 (2002)
Whitaker, C., Kuncheva, L.: Measures of diversity in classifier ensembles. Machine Learning 51, 181–207 (2003)
Prechelt, L.: Proben1 - a set of benchmarks and benchmarking rules for neural training algorithms, Tech. Report 21/94, Universitat Karlsruhe (1994)
Allende, H., Moraga, C., Ñanculef, R., Valle, C.: Self-poised ensemble learning. In: Famili, A.F., Kok, J.N., Peña, J.M., Siebes, A., Feelders, A. (eds.) IDA 2005. LNCS, vol. 3646, pp. 272–282. Springer, Heidelberg (2005)
Rosen, B.: Ensemble learning used decorrelated neural networks. Connection Science 8(3-4), 373–384 (1999)
Schapire, R.: The strength of weak learnability. Machine Learning 5, 197–227 (1990)
Yao, X., Lui, Y.: Ensemble learning via negative correlation. Neural Networks 12(10), 1399–1404 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ñanculef, R., Valle, C., Allende, H., Moraga, C. (2005). Moderated Innovations in Self-poised Ensemble Learning. In: Hao, Y., et al. Computational Intelligence and Security. CIS 2005. Lecture Notes in Computer Science(), vol 3801. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11596448_7
Download citation
DOI: https://doi.org/10.1007/11596448_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-30818-8
Online ISBN: 978-3-540-31599-5
eBook Packages: Computer ScienceComputer Science (R0)