Abstract
How to acquire new knowledge from new added training data while retaining the knowledge learned before is an important problem for incremental learning. In order to handle this problem, we propose a novel algorithm that enables support vector machines to accommodate new data, including samples that correspond to previously unseen classes, while it retains previously acquired knowledge. Furthermore, our new algorithm does not require access to previously used data during subsequent incremental learning sessions. The proposed algorithm trains a support vector machine that can output posterior probability information once an incremental batch training data is acquired. The outputs of all the resulting support vector machines are simply combined by averaging. Experiments are carried out on three benchmark datasets as well as a real world text categorization task. The experimental results indicate that the proposed algorithm is superior to the traditional incremental learning algorithm, Learn++. Due to the simplicity of the proposed algorithm, it can be used more effectively in practice.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Jantke, P.: Types of Incremental Learning. In: AAAI Symposium on Training Issues in Incremental Learning, Standford, CA, March 23-25, 1993, AAAI Press, Menlo Park (1993)
Zhou, Z.H., Chen, Z.Q.: Hybrid Decisions Tree. Knowledge-Based System 15, 515–528 (2002)
Syed, N.A., Huan, L., Sung, K.K.: Handling Concept Drifts in Incremental Learning with Support Vector Machines. In: Proceedings of KDD-99, San Diego, CA, USA (1999)
Grossberg, S.: Nonlinear Neural Networks: Principles, Mechanisms and Architectures. Neural Networks 1, 17–61 (1988)
Rüping, S.: Incremental Learning with Support Vector Machines. In: Proceedings of the IEEE International Conference on Data Mining, San Jose, CA, IEEE Computer Society Press, Los Alamitos (2001)
Lu, B.L., Ito, M.: Task Decomposition and Module Combination Based on Class Relations: a Modular Neural Networks for Pattern Classification. IEEE Transaction on Neural Networks 10, 1244–1256 (1999)
Zhou, Z.H., Chen, S.F.: Neural Network Ensemble (in Chinese). Chinese J. Computers 25, 1–8 (2002)
Xu, L., Krzyżak, A., Suen, C.Y.: Methods of Combining Multiple Classifiers and Their Application to Handwriting Recognition. IEEE Transaction on Systems, Man, and Cybernetics 22, 418–434 (1992)
Polikar, R., et al.: Learn++: An Incremental Learning Algorithm for Supervised Neural Networks. IEEE Transaction on Systems, Man, and Cybernetics 31, 497–508 (2001)
Lu, B.L., Ichikawa, M.: Emergent Online Learning in Min-max Modular Neural Networks. In: Proceedings of IJCNN’01, pp. 2650–2655 (2001)
Macek, J.: Incremental Learning of Ensemble Classifiers on ECG data. In: Proceedings of CBMS’05 (2005)
Wang, H.X., et al.: Mining Concept-drifting Data Streams Using Ensemble Classifiers. In: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, ACM Press, New York (2003)
Breiman, L.: Bagging Predictors. Machine Learning 24, 123–140 (1996)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Wen, YM., Lu, BL. (2007). Incremental Learning of Support Vector Machines by Classifier Combining. In: Zhou, ZH., Li, H., Yang, Q. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2007. Lecture Notes in Computer Science(), vol 4426. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71701-0_101
Download citation
DOI: https://doi.org/10.1007/978-3-540-71701-0_101
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-71700-3
Online ISBN: 978-3-540-71701-0
eBook Packages: Computer ScienceComputer Science (R0)