Abstract
A fast support vector machine (SVM) training algorithm is proposed under the decomposition framework of SVM’s algorithm by effectively integrating kernel caching, digest and shrinking policies and stopping conditions. Extensive experiments on MNIST handwritten digit database have been conducted to show that the proposed algorithm is much faster than Keerthi et al.’s improved SMO, about 9 times. Combined with principal component analysis, the total training for ten one-against-the-rest classifiers on MNIST took just 0.77 hours. The promising scalability of the proposed scheme can make it possible to apply SVM to a wide variety of problems in engineering.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
C. M. Bishop. Neural Networks for Pattern Recognition. Clarendon Press, Oxford, 1995.
C. J. C. Burges. A tutorial on support vector machines for pattern recognition. In Data mining and Knowledge Discovery, pages 121–167. 1998.
C. J. C. Burges and D. J. Crisp. Uniqueness of the svm solution. In Advances in Neural Information Processing Systems, 2001. to appear in NIPS 12.
N. Cristianini and J. S. Taylor. An Introduction to Support Vector Machines and other kernel-based learning methods. Cambridge University Press, 2000.
D. DeCoste and B. Schölkopf. Training invariant support vector machines. Machine Learning, 46(1–3):161–190, 2002.
J. X. Dong, C. Y. Suen, and A. Krzyżak. A fast svm training algorithm. Technical report, CENPARMI, Concordia University, Montréal, Canada, December 2002.
J. X. Dong, C. Y. Suen, and A. Krzyżak. A practical smo algorithm. In Proceedings of the International Conference on Pattern Recognition, Quebec City, Canada, August 2002.
G. W. Flake and S. Lawrence. Efficient svm regression training with smo. Machine Learning, 46(1–3):271–290, March 2002.
Y. Fujisawa, M. Shi, T. Wakabayashi, and F. Kimura. Handwritten numeral recognition using gradient and curvature of gray scale image. In Proceedings of International Conference on Document Analysis and Recognition, pages 277–280, India, August 1999.
T. Joachims. Making large-scale support vector machine learning practical. In B. Schölkopf, C. Burges, and A. Smola, editors, Advances in kernel methods: Support Vector Machines. MIT Press, Cambridge, MA, December 1998.
T. Joachims. Text categorization with support vector machine. In Proceedings of European Conference on Machine Learning(ECML), 1998.
S. S. Keerthi and E. G. Gilbert. Convergence of a generalized smo algorithm for svm classifier design. Machine Learning, 46(3):351–360, March 2002.
S. S. Keerthi, S. K. Shevade, C. Bhattachayya, and K. R. K. Murth. Improvements to platt’s smo algorithm for svm classifier design. Neural Computation, 13:637–649, March 2001.
H. Kuhn and A. Tucker. Nonlinear programming. In Proceedings of 2nd Berkeley Symposium on Mathematical Statistics and Probabilistics, pages 481–492. University of California Press, 1951.
C.-J. Lin. Formulations of support vector machines: A note from an optimization point of view. Neural Computation, 13:307–317, 2001.
J. Mercer. Functions of positive and negative type and their connection with the theory of integral equations. Philos. Trans. Roy. Soc. London, A(209):415–446, 1909.
E. Osuna, R. Freund, and F. Girosi. Training support vector machines: An application to face detection. In Proceedings of the 1997 conference on Computer Vision and Pattern Recognition(CVPR’97), Puerto Rico, June 17–19 1997.
J. C. Platt. Fast training of support vector machines using sequential minimial optimization. In B. Schölkopf, C. Burges, and A. Smola, editors, Advances in kernel methods: Support Vector Machines. MIT Press, Cambridge, MA, December 1998.
B. Schölkopf, C. J. C. Burges, and V. Vapnik. Extracting support data for a given task. In U. M. Fayyad and R. Uthurusamy, editors, Proceedings, First International Conference on Knowledge Discovery and Data Mining, pages 252–257. AAAI Press, Menlo Park, CA, 1995.
V. N. Vapnik. Statistical Learning Theory. Wiley, New York, 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dong, Jx., Krzyżak, A., Suen, C.Y. (2002). A Fast SVM Training Algorithm. In: Lee, SW., Verri, A. (eds) Pattern Recognition with Support Vector Machines. SVM 2002. Lecture Notes in Computer Science, vol 2388. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45665-1_5
Download citation
DOI: https://doi.org/10.1007/3-540-45665-1_5
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44016-1
Online ISBN: 978-3-540-45665-0
eBook Packages: Springer Book Archive