Abstract
Support Vector Machines (SVMs) with few support vectors are quite desirable, as they have a fast application to new, unseen patterns. In this work we shall study the coefficient structure of the dual representation of SVMs constructed for nonlinearly separable problems through kernel perceptron training. We shall relate them with the margin of their support vectors (SVs) and also with the number of iterations in which these SVs take part. These considerations will lead to a remove–and–retrain procedure for building SVMs with a small number of SVs where both suitably small and large coefficient SVs will be taken out from the training sample. Besides providing a significant SV reduction, our method’s computational cost is comparable to that of a single SVM training.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bennett, K., Bredensteiner, E.: Geometry in learning. In: Gorini, C., Hart, E., Meyer, W., Phillips, T. (eds.) Geometry at Work, Mathematical Association of America, Washington D.C (1997)
Burges, C.J.C.: Simplified support vector decision rules. In: Saitta, L. (ed.) Proc. 13th International Conference on Machine Learning, pp. 71–77. Morgan Kaufmann, San Francisco (1996)
Chang, C., Lin, C.: LIBSVM: a library for support vector machines (2001), http://www.csie.ntu.edu.tw/~cjlin/LIBSVM
Dekel, O., Shalev-Shwartz, S., Singer, Y.: The Forgetron: A Kernel-Based Perceptron on a Fixed Budget. Advances in Neural Processing Systems 18, 259–266 (2005)
Franc, V., Hlavac, V.: An iterative algorithm learning the maximal margin classier. Pattern Recognition 36, 1985–1996 (2003)
García, D., González, A., Dorronsoro, J.R.: Convex Perceptrons. In: Corchado, E.S., Yin, H., Botti, V., Fyfe, C. (eds.) IDEAL 2006. LNCS, vol. 4224, pp. 578–585. Springer, Heidelberg (2006)
Joachims, T.: Making Large-Scale Support Vector Machine Learning Practical. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel methods, pp. 169–184. MIT Press, Cambridge (1999)
Lee, Y., Mangasarian, O.L.: RSVM: reduced support vector machines. In: CD Proceedings of the First SIAM International Conference on Data Mining, Chicago (2001)
Keerthi, S., Chapelle, O., de Coste, D.: Building support vector machines with reduced complexity. Journal of Machine Learning Research 7, 1493–1515 (2006)
Parrado-Hernéndez, E., Mora-Jiménez, I., Arenas-García, J., Figueiras-Vidal, A.R., Navia-Vázquez, A.: Growing support vector classifiers with controlled complexity. Pattern Recognition 36, 1479–1488 (2003)
Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel methods, pp. 185–208. MIT Press, Cambridge (1999)
Schlesinger, M., Hlavac, V.: Ten Lectures on Statistical and Structural Pattern Recognition. Kluwer Academic Publishers, Dordrecht (2002)
Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2001)
UCI-benchmark repository of machine learning data sets. University of California Irvine, http://www.ics.uci.edu
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, Berlin (1995)
Wu, M., Schölkopf, B., Bakir, G.: A Direct Method for Building Sparse Kernel Learning Algorithms. Journal of Machine Learning Research 7, 603–624 (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
García, D., González, A., Dorronsoro, J.R. (2007). Coefficient Structure of Kernel Perceptrons and Support Vector Reduction. In: Mira, J., Álvarez, J.R. (eds) Bio-inspired Modeling of Cognitive Tasks. IWINAC 2007. Lecture Notes in Computer Science, vol 4527. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73053-8_34
Download citation
DOI: https://doi.org/10.1007/978-3-540-73053-8_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73052-1
Online ISBN: 978-3-540-73053-8
eBook Packages: Computer ScienceComputer Science (R0)