Abstract
Trained support vector machines (SVMs) have a slow run-time classification speed if the classification problem is noisy and the sample data set is large. Approximating the SVM by a more sparse function has been proposed to solve to this problem. In this study, different variants of approximation algorithms are empirically compared. It is shown that gradient descent using the improved Rprop algorithm increases the robustness of the method compared to fixed-point iteration. Three different heuristics for selecting the support vectors to be used in the construction of the sparse approximation are proposed. It turns out that none is superior to random selection. The effect of a finishing gradient descent on all parameters of the sparse approximation is studied.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Steinwart, I.: Sparseness of support vector machines. Journal of Machine Learning Research 4, 1071–1105 (2003)
Burges, C.J.C.: Simplified support vector decision rules. In: ICML 1996. Proceedings of the 13th International Conference on Machine Learning, pp. 71–77 (1996)
Burges, C.J.C., Schölkopf, B.: Improving the accuracy and speed of support vector machines. In: Mozer, M., Jordan, M., Petsche, T. (eds.) Advances in Neural Information Processing Systems, Cambridge, MA, vol. 9, pp. 375–381 (1997)
Schölkopf, B., Knirsch, P., Smola, A.J., Burges, C.J.C.: Fast approximation of support vector kernel expansions, and an interpretation of clustering as approximation in feature space. In: Levi, P., Ahlers, R.J., May, F., Schanz, M. (eds.) DAGM-Symposium, pp. 124–132. Springer, Heidelberg (1998)
Schölkopf, B., Mika, S., Burges, C.J.C., Knirsch, P., Müller, K.R., Rätsch, G., Smola, A.J.: Input space versus feature space in kernel-based methods. IEEE Transactions on Neural Networks 10(5), 1000–1017 (1999)
Romdhani, S., Torr, P., Schölkopf, B., Blake, A.: Efficient face detection by a cascaded support-vector machine expansion. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 460, 3283–3297 (2004)
Igel, C., Hüsken, M.: Empirical evaluation of the improved Rprop learning algorithm. Neurocomputing 50(C), 105–123 (2003)
Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20(3), 273–297 (1995)
Glasmachers, T., Igel, C.: Maximum-gain working set selection for support vector machines. Journal of Machine Learning Research 7, 1437–1466 (2006)
Baker, J.E.: Reducing bias and inefficiency in the selection algorithm. In: Grefenstette, J.J. (ed.) Proceedings of the Second International Conference on Genetic Algorithms, pp. 14–21 (1987)
Girolami, M.: Mercer kernel-based clustering in feature space. IEEE Transactions on Neural Networks 13(3), 780–784 (2002)
Zhang, R., Rudnicky, A.I.: A large scale clustering scheme for kernel k-means. In: Proceedings of the International Conference on Pattern Recognition, pp. 289–292 (2002)
Riedmiller, M.: Advanced supervised learning in multi-layer perceptrons – From backpropagation to adaptive learning algorithms. Computer Standards and Interfaces 16(5), 265–278 (1994)
Rätsch, G., Onoda, T., Müller, K.–R.: Soft margins for AdaBoost. Machine Learning 42(3), 287–320 (2001)
Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Suttorp, T., Igel, C. (2007). Resilient Approximation of Kernel Classifiers. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D. (eds) Artificial Neural Networks – ICANN 2007. ICANN 2007. Lecture Notes in Computer Science, vol 4668. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74690-4_15
Download citation
DOI: https://doi.org/10.1007/978-3-540-74690-4_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74689-8
Online ISBN: 978-3-540-74690-4
eBook Packages: Computer ScienceComputer Science (R0)