Abstract
We propose a technique for a training set approximation and its usage in kernel methods. The approach aims to represent data in a low dimensional space with possibly minimal representation error which is similar to the Principal Component Analysis (PCA). In contrast to the PCA, the basis vectors of the low dimensional space used for data representation are properly selected vectors from the training set and not as their linear combinations. The basis vectors can be selected by a simple algorithm which has low computational requirements and allows on-line processing of huge data sets. The proposed method was used to approximate training sets of the Support Vector Machines and Kernel Fisher Linear Discriminant which are known method for learning classifiers. The experiments show that the proposed approximation can significantly reduce the complexity of the found classifiers (the number of the support vectors) while retaining their accuracy.
The authors were supported by the European Union projects ICA 1-CT-2000- 70002, IST-2001-32184 ActIpret, by the Czech Ministry of Education under project MSM 212300013, by the Grant Agency of the Czech Republic project 102/03/0440. The authors would like to thank to the anonymous reviewers for their useful comments.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Intelligent Data Analysis (IDA) repository, http://ida.first.gmd.de/~raetsch
Burges, C.J.C.: Simplified Support Vector Decision Rule. In: 13th Intl. Conf. on Machine Learning, San Mateo, pp. 71–77. Morgan Kaufmann, San Francisco (1996)
Golub, G.H., van Loan, C.F.: Matrix Computations, 3rd edn. John Hopkins University Press, Baltimore (1996)
Mika, S., Rätsch, G., Müller, K.R.: A Mathematical Programming Approach to the Kernel Fisher Algorithm. In: NIPS, pp. 591–597 (2000)
Mika, S., Rätsch, G., Weston, J., Schölkopf, B., Müller, K.R.: Fisher Discriminant Analysis with Kernels. In: Hu, Y.-H., Larsen, J., Wilson, E., Douglas, S. (eds.) Neural Networks for Signal Processing IX, pp. 41–48. IEEE, Los Alamitos (1999)
Mika, S., Smola, A., Scholkopf, B.: An Improved Training Algorithm for Kernel Fisher Discriminants. In: AISTATS 2001. Morgan Kaufmann, San Francisco (2001)
Osuna, E., Girosi, F.: Reducing the Runtime Complexity in Support Vector Machines. In: Advances in Kernel Methods, pp. 271–284. MIT Press, Cambridge (1998)
Platt, J.C.: Sequential Minimal Optimizer: A Fast Algorithm for Training Support Vector Machines. Technical Report MSR-TR-98-14, Microsoft Research (1998)
Schölkopf, B., Knirsch, P., Smola, C., Burges, A.: Fast Approximation of Support Vector Kernel Expansions, and an Interpretation of Clustering as Approximation in Feature Spaces. In: Ahler, R.-J., Levi, P., Schanz, M., May, F. (eds.) Mustererkennung 1998-20. DAGM-Symp., Berlin, Germany, pp. 124–132. Springer, Heidelberg (1998)
Schölkopf, B., Smola, A., Müller, K.R.: Nonlinear Component Analysis as a Kernel Eigenvalue Problem. Technical report, Max-Planck-Institute fur biologische Kybernetik (1996)
Vapnik, V.: Statistical Learning Theory. John Wiley & Sons, Inc., Chichester (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Franc, V., Hlaváč, V. (2003). Greedy Algorithm for a Training Set Reduction in the Kernel Methods. In: Petkov, N., Westenberg, M.A. (eds) Computer Analysis of Images and Patterns. CAIP 2003. Lecture Notes in Computer Science, vol 2756. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45179-2_53
Download citation
DOI: https://doi.org/10.1007/978-3-540-45179-2_53
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40730-0
Online ISBN: 978-3-540-45179-2
eBook Packages: Springer Book Archive