Abstract
The computational difficulties occurred when we use a conventional support vector machine with nonlinear kernels to deal with massive datasets. The reduced support vector machine (RSVM) replaces the fully dense square kernel matrix with a small rectangular kernel matrix which is used in the nonlinear SVM formulation to avoid the computational difficulties. In this paper, we propose a new algorithm, Systematic Sampling RSVM (SSRSVM) that selects the informative data points to form the reduced set while the RSVM used random selection scheme. This algorithm is inspired by the key idea of SVM, the SVM classifier can be represented by support vectors and the misclassified points are a part of support vectors. SSRSVM starts with an extremely small initial reduced set and adds a portion of misclassified points into the reduced set iteratively based on the current classifier until the validation set correctness is large enough. In our experiments, we tested SSRSVM on six public available datasets. It turns out that SSRSVM might automatically generate a smaller size of reduced set than the one by random sampling. Moreover, SSRSVM is faster than RSVM and much faster than conventional SVM under the same level of the test set correctness.
Keywords
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2(2), 121–167 (1998)
Center, M.I.T.: for Biological and Computation Learning. Cbcl face database (1) (2000), http://www.ai.mit.edu/projects/cbcl
Fung, G., Mangasarian, O.L.: Proximal support vector machine classifiers. In: Provost, F., Srikant, R. (eds.) Proceedings KDD-2001: Knowledge Discovery and Data Mining, San Francisco, CA, August 26-29, pp. 77–86. Asscociation for Computing Machinery, New York (2001), ftp://ftp.cs.wisc.edu/pub/dmi/techreports/01-02.ps
Lee, Y.-J., Hsieh, W.-F.: ε-ssvr: A smooth support vector machine for ε-insensitive regression. IEEE Transactions on Knowledge and Data Engineering (2003) (submitted)
Lee, Y.-J., Mangasarian, O.L.: RSVM: Reduced support vector machines. Technical Report 00-07, Data Mining Institute, Computer Sciences Department, University of Wisconsin, Madison, Wisconsin (July 2000); Proceedings of the First SIAM International Conference on Data Mining, Chicago, April 5-7 (2001), CD-ROM Proceedings, ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/00-07.ps
Lee, Y.-J., Mangasarian, O.L.: SSVM: A smooth support vector machine. Computational Optimization and Applications 20, 5–22 (2001); Data Mining Institute, University of Wisconsin, Technical Report 99-03, ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/99-03.ps
Mangasarian, O.L.: Generalized support vector machines. In: Smola, A., Bartlett, P., Schölkopf, B., Schuurmans, D. (eds.) Advances in Large Margin Classifiers, pp. 135–146. MIT Press, Cambridge (2000), ftp://ftp.cs.wisc.edu/math-prog/tech-reports/98-14.ps
Murphy, P.M., Aha, D.W.: UCI machine learning repository (1992), www.ics.uci.edu/~mlearn/MLRepository.html
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chang, CC., Lee, YJ. (2004). Generating the Reduced Set by Systematic Sampling. In: Yang, Z.R., Yin, H., Everson, R.M. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2004. IDEAL 2004. Lecture Notes in Computer Science, vol 3177. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-28651-6_107
Download citation
DOI: https://doi.org/10.1007/978-3-540-28651-6_107
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22881-3
Online ISBN: 978-3-540-28651-6
eBook Packages: Springer Book Archive