Abstract
Support vector machine (SVM) provides good generalization performance but suffers from a large amount of computation. This paper presents an incremental learning strategy for support vector regression (SVR). The new method firstly formulates an explicit expression of ||W||2 by constructing an orthogonal basis in feature space together with a basic Hilbert space identity, and then finds the regression function through minimizing the formula of ||W||2 rather than solving a convex programming problem. Particularly, we combine the minimization of ||W||2 with kernel selection that can lead to good generalization performance. The presented method not only provides a novel way for incremental SVR learning, but opens an opportunity for model selection of SVR as well. An artificial data set, a benchmark data set and a real-world data set are employed to evaluate the method. The simulations support the feasibility and effectiveness of the proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
O. Barzilay V. Brailovsky (1999) ArticleTitleOn domain knowledge and feature selection using a support vector machine Pattern Recognition Letters 20 475–484
Blake, C. L. and Merz, C. J.: UCI Repository of machine learning databases, University of California, Department of Information and Computer Science. 1998. http://www.ics.uci.deu/~mlearn/MLRRepository.html
Brown, M., Grundy, W., Lin, D., Cristianini, N., Sugnet, C., Furey, T., Ares, M. and Haussler, D.: Knowledge-based analysis of microarray gene expression data by using support vector machines, In: Proceedings of The National Academy of Sciences of The United States of America, 97: 262–267, (2000)
Campbell, C., Cristianini, N. and Smola, A.: Query learning with large margin classifiers, In: Proceedings 17th International Conference on Machine Learning. 111–118, San Francisco, CA: Morgan Kaufmann, (2000)
Cauwenberghs, G. and Poggio, T.: Incremental and decremental support vector machine learning. Advances in Neural Information Processing Systems, 13: MIT Press, (2001)
O. Chapelle V. Vapnik (2000) Model selection for support vector machines A. Smola T. Leen K. Müller (Eds) Advances in Neural Information Processing Systems 12 MIT Press Cambridge, MA 230–236
Cristianini, N. and Shawe-Taylor, J.: An Introduction to Support Vector Machines, Cambridge University Press, 2000
H. Drucker D. Wu V. Vapnik (1999) ArticleTitleSupport vector machines for span categorization IEEE Trans. Neural Networks 10 1048–1054
Fine, S. and Scheinberg, K.: Incremental Learning and Selective Sampling via Parametric Optimization Framework for SVM. In Machine Learning, Kluwer Academic, 2002.
Frieß, T.-T. Chistianini, N. and Campbell, C.: The kernel adatron algorithm: A fast and simple learning procedure for support vector machines. In: J. Shavlik (ed.), Proceedings of the The 15th International Conference of Machine Learning. San Francisco, 188–196, CA: Morgan Kaufmann (1998)
Gunn, S. R.: Support vector machines for classification and regression, Technical Report, Image Speech and Intelligent System Research Group, University of Southampton, 1997
Joachims, T.: Text categorization with support vector machines: Learning with many relevant features, In: Proceedings of the European Conference on Machine Learning, 137–142, Berlin: Springer (1998)
T. Joachims (1998) Make large-scale support vector machine learning practical. Advances in kernel methods: support vector machines MIT Press Cambridge, MA 169–184
Martin, M.: On-line support vector machines for function approximation, (2002) http://www.lsi.upc.es/dept/techreps/html/R02-11.html
Osuna, E. Freund, R. and Girosi, F.: An improved training algorithm for support vector machines. Proceedings of the IEEE Workshop on Neural Networks for Signal Processing, 276–285, (1997)
J.C. Platt (1998) Fast training of support vector machines using sequential minimum optimization, Advances in kernel methods: support vector machines MIT Press Cambridge, MA 185–208
T. Poggio F. Girosi (1990) ArticleTitleNetworks for approximation and learning Proceedings on IEEE 78 IssueID9 1481–1497
V. Vapnik (1995) The Nature of Statistical Learning Theory John Wiley & Sons New York
V. Vapnik (1998) Statistical Learning Theory John Wiley & Sons New York
V. Vapnik S. Golowich A. Smola (1997) Support Vector Method for Function Approximation, Regression Estimation, and Signal Processing M. Mozer M. Jordan T. Petsche (Eds) Neural Information Processing Systems MIT Press Cambridge, MA
W. J. Wang Z. B. Xu W. Z. Lu (2003) ArticleTitleThree improved neural network models for air quality forecasting Engineering Computations 20 IssueID2 192–210
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
WANG, W. An Incremental Learning Strategy for Support Vector Regression. Neural Process Lett 21, 175–188 (2005). https://doi.org/10.1007/s11063-004-5714-1
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s11063-004-5714-1