skip to main content
10.1145/1102351.1102429acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicmlConference Proceedingsconference-collections
Article

An efficient method for simplifying support vector machines

Published: 07 August 2005 Publication History

Abstract

In this paper we describe a new method to reduce the complexity of support vector machines by reducing the number of necessary support vectors included in their solutions. The reduction process iteratively selects two nearest support vectors belonging to the same class and replaces them by a newly constructed vector. Through the analysis of relation between vectors in the input and feature spaces, we present the construction of new vectors that requires to find the unique maximum point of a one-variable function on the interval (0, 1), not to minimize a function of many variables with local minimums in former reduced set methods. Experimental results on real life datasets show that the proposed method is effective in reducing number of support vectors and preserving machine's generalization performance.

References

[1]
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. Proceedings of the Fifth Annual Workshop on Computational Learning Theory (pp. 144--152). Pittsburgh, Pennsylvania, United States.
[2]
Burges, C. J. C. (1996). Simplified support vector decision rules. Proc. 13th International Conference on Machine Learning (pp. 71--77). San Mateo, CA.
[3]
Burges, C. J. C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2, 121--167.
[4]
Burges, C. J. C., & Schoelkopf, B. (1997). Improving the accuracy and speed of support vector learning machines. In M. Mozer, M. Jordan and T. Petsche (Eds.), Advances in neural information processing systems 9, 375--381. Cambridge, MA: MIT Press.
[5]
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20, 273--297.
[6]
Cristianini, C., & Shawe-Taylor, J. (2000). An introduction to support vector machines. Cambridge University Press.
[7]
DeCoste, D., & Mazzoni, D. (2003). Fast query-optimized kernel machine classification via incremental approximate nearest support vectors. Proceedings International Conference on Machine Learning (ICML-03) (pp. 115--122).
[8]
Joachims, T. (1998). Text categorization with support vector machines: Learning with many relevant features. Proceedings of the European Conference on Machine Learning (pp. 137--142). Berlin: Springer.
[9]
Kwok, J., & Tsang, I. (2003). The pre-image problem in kernel methods. In Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003) (pp. 408--415). Washington, D.C., USA.
[10]
LeCun, Y., Botou, L., Jackel, L., Drucker, H., Cortes, C., Denker, J., Guyon, I., Muller, U., Sackinger, E., Simard, P., & Vapnik, V. (1995). Learning algorithms for classification: A comparison on handwritten digit recognition. Neural Networks, 261--276.
[11]
Liu, C., Nakashima, K., Sako, H., & Fujisawa, H. (2003). Handwritten digit recognition: benchmarking of state-of-the-art techniques. Pattern Recognition, 36, 2271--2285.
[12]
Mika, S., Schoelkopf, B., Smola, A., Muller, K.-R., Scholz, M., & Ratsch, G. (1999). Kernel pea and de-noising in feature spaces. In M. S. Kearns, S. A. Solla and D. A. Cohn (Eds.), Advances in neural information processing systems 11, 536--542. Cambridge, MA: MIT Press.
[13]
Osuna, E., Freund, R., & Girosi, F. (1997). Training support vector machines:an application to face detection. IEEE Conference on Computer Vision and Pattern Recognition (pp. 130--136). Puerto Rico.
[14]
Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (2002). Numerical recipes in c++: the art of scientific computing. Cambridge University Press.
[15]
Schoelkopf, B., Mika, S., Burges, C. J. C., Knirsch, P., Muller, K., Ratsch, G., & Smola, A. J. (1999). Input space versus feature space in kernel-based methods. IEEE Trans. Neural Networks, 10, 1000--1017.
[16]
Schoelkopf, B., & Smola, A. (2002). Learning with kernels. Cambridge, MA: MIT Press.
[17]
Vapnik, V. (1995). The nature of statistical learning theory. N.Y.: Springer.

Cited By

View all
  • (2023)Terrain Type Detection for Smart Equine Gait Analysis Systems Using Inertial Sensors and Machine Learning2023 19th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT)10.1109/DCOSS-IoT58021.2023.00029(103-111)Online publication date: Jun-2023
  • (2022)In-Field Detection of American Foulbrood (AFB) by Electric Nose Using Classical Classification Techniques and Sequential Neural NetworksSensors10.3390/s2203114822:3(1148)Online publication date: 2-Feb-2022
  • (2021)Support vector machine and its difficulties from control field of viewTransactions of the Institute of Measurement and Control10.1177/014233122097743643:9(1833-1842)Online publication date: 25-Feb-2021
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICML '05: Proceedings of the 22nd international conference on Machine learning
August 2005
1113 pages
ISBN:1595931805
DOI:10.1145/1102351
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 August 2005

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Article

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)4
  • Downloads (Last 6 weeks)1
Reflects downloads up to 06 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Terrain Type Detection for Smart Equine Gait Analysis Systems Using Inertial Sensors and Machine Learning2023 19th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT)10.1109/DCOSS-IoT58021.2023.00029(103-111)Online publication date: Jun-2023
  • (2022)In-Field Detection of American Foulbrood (AFB) by Electric Nose Using Classical Classification Techniques and Sequential Neural NetworksSensors10.3390/s2203114822:3(1148)Online publication date: 2-Feb-2022
  • (2021)Support vector machine and its difficulties from control field of viewTransactions of the Institute of Measurement and Control10.1177/014233122097743643:9(1833-1842)Online publication date: 25-Feb-2021
  • (2020)Online Budgeted Stochastic Coordinate Ascent for Large-Scale Kernelized Dual Support Vector Machine TrainingPattern Recognition Applications and Methods10.1007/978-3-030-40014-9_2(23-47)Online publication date: 25-Jan-2020
  • (2019)Sliding Mode Control based Support Vector Machine RBF Kernel Parameter Optimization2019 IEEE International Conference on Imaging Systems and Techniques (IST)10.1109/IST48021.2019.9010479(1-6)Online publication date: Dec-2019
  • (2019)Purity Filtering: An Instance Selection Method for Support Vector MachinesArtificial Intelligence XXXVI10.1007/978-3-030-34885-4_2(21-35)Online publication date: 19-Nov-2019
  • (2019)Speeding Up Budgeted Stochastic Gradient Descent SVM Training with Precomputed Golden Section SearchMachine Learning, Optimization, and Data Science10.1007/978-3-030-13709-0_28(329-340)Online publication date: 14-Feb-2019
  • (2018)A Fast Algorithm of Convex Hull Vertices Selection for Online ClassificationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2017.264803829:4(792-806)Online publication date: Apr-2018
  • (2018)A Modification of Solution Optimization in Support Vector Machine Simplification for ClassificationInformation Systems Design and Intelligent Applications10.1007/978-981-10-7512-4_15(149-158)Online publication date: 2-Mar-2018
  • (2017)A Comparison of SimpSVM and RVM for Sign Language RecognitionProceedings of the 2017 International Conference on Machine Learning and Soft Computing10.1145/3036290.3036322(98-104)Online publication date: 13-Jan-2017
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media