Abstract
Classifier ensemble is a main direction of incremental learning researches, and many ensemble-based incremental learning methods have been presented. Among them, Learn++, which is derived from the famous ensemble algorithm, AdaBoost, is special. Learn++ can work with any type of classifiers, either they are specially designed for incremental learning or not, this makes Learn++ potentially supports heterogeneous base classifiers. Based on massive experiments we analyze the advantages and disadvantages of Learn++. Then a new ensemble incremental learning method, Bagging++, is presented, which is based on another famous ensemble method: Bagging. The experimental results show that Bagging ensemble is a promising method for incremental learning and heterogeneous Bagging++ has the better generalization and learning speed than other compared methods such as Learn++ and NCL.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Giraud-Carrier, C.: A Note on the Utility of Incremental Learning. AI Communications 13(4), 215–223 (2000)
Polikar, R., Udpa, L., Udpa, S.S., Honavar, V.: Learn++: An incremental learning algorithm for supervised neural networks. IEEE Transactions on Systems, Man, and Cybernetics - Part C: Applications and Reviews 31(4), 497–508 (2001)
Utgoff, P.E.: Incremental induction of decision trees. Machine Learning 4, 161–186 (1989)
Cauwenberghs, G., Poggio, T.: Incremental and Decremental Support Vector Machine Learning. In: Advances in Neural Information Processing Systems, vol. 12, pp. 409–415. MIT Press, Cambridge (2000)
Carpenter, G.A., Grossberg, S., Reynolds, J.H.: ARTMAP: Supervied real-time learning and classification of nonstationary data by a self organizing neural network. Neural Networks 4(5), 565–588 (1991)
Kasabov, N.: Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning. IEEE Transactions on Systems, Man and Cybernetics - Part B: Cybernetics 31(6), 902–918 (2001)
Sewell, M.: Ensemble learning (2008), http://machine-learning.martinsewell.com/ensembles/ensemble-learning.pdf, (unpublished)
Seipone, T., Bullinaria, J.: Evolving improved incremental learning schemes for neural network systems. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computing (CEC 2005), Piscataway, NJ, pp. 273–280 (2005)
Inoue, H., Narihisa, H.: Self-organizing neural grove and its applications. In: Proceedings of the 2005 International Joint Conference on Neural Networks (IJCNN 2005), Montreal, Canada, pp. 1205–1210 (2005)
Minku, F.L., Inoue, H., Yao, X.: Negative correlation in incremental learning. Natural Computing 8, 280–320 (2009)
Breiman, L.: Bagging Predictors. Machine Learning 24(2), 123–140 (1996)
Schwenk, H., Bengio, Y.: Boosting neural networks. Neural Computation 12, 1869–1887 (2000)
Asuncion, D.N.A.: UCI machine learning repository (2007), http://www.ics.uci.edu/mlearn/MLRepository.html (unpublished)
Tang, K., Lin, M., Minku, F.L., Yao, X.: Selective Negative Correlation Learning Approach to Incremental Learning. Neurocomputing 72(13-15), 2796–2805 (2009)
Riedmiller, M., Braun, H.: RPROP- A fast adaptive learning algorithm. In: Proc. of ISCIS VII (1992)
Lin, C.J.: LIBSVM: A Library for Support Vector Machines (2009), http://www.csie.ntu.edu.tw/~cjlin/libsvm/ (unpublished)
Demsar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)
Maloof, M.A., Michalski, R.S.: Incremental learning with partial instance memory. Artificial Intelligence 154(1-2), 95–126 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhao, Q.L., Jiang, Y.H., Xu, M. (2010). Incremental Learning by Heterogeneous Bagging Ensemble. In: Cao, L., Zhong, J., Feng, Y. (eds) Advanced Data Mining and Applications. ADMA 2010. Lecture Notes in Computer Science(), vol 6441. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17313-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-17313-4_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17312-7
Online ISBN: 978-3-642-17313-4
eBook Packages: Computer ScienceComputer Science (R0)