Article

Simpler core vector machines with enclosing balls

Authors:

James T. KwokAuthors Info & Claims

ICML '07: Proceedings of the 24th international conference on Machine learning

Pages 911 - 918

https://doi.org/10.1145/1273496.1273611

Published: 20 June 2007 Publication History

Abstract

The core vector machine (CVM) is a recent approach for scaling up kernel methods based on the notion of minimum enclosing ball (MEB). Though conceptually simple, an efficient implementation still requires a sophisticated numerical solver. In this paper, we introduce the enclosing ball (EB) problem where the ball's radius is fixed and thus does not have to be minimized. We develop efficient (1 + e)-approximation algorithms that are simple to implement and do not require any numerical solver. For the Gaussian kernel in particular, a suitable choice of this (fixed) radius is easy to determine, and the center obtained from the (1 + e)-approximation of this EB problem is close to the center of the corresponding MEB. Experimental results show that the proposed algorithm has accuracies comparable to the other large-scale SVM implementations, but can handle very large data sets and is even faster than the CVM in general.

References

[1]

Asharaf, S., Murty, M., & Shevade, S. (2006). Cluster based core vector machine. Proceedings of the Sixth International Conference on Data Mining (pp. 1038--1042).

Digital Library

[2]

Bordes, A., Ertekin, S., Weston, J., & Bottou, L. (2005). Fast kernel classifiers with online and active learning. Journal of Machine Learning Research, 6, 1579--1619.

Digital Library

[3]

Chapelle, O. (2007). Training a support vector machine in the primal. Neural Computation, 19, 1155--1178.

Digital Library

[4]

Crammer, K., Dekel, O., Keshet, J., Shalev-Shwartz, S., & Singer, Y. (2006). Online passive-aggressive algorithms. Journal of Machine Learning Research, 7, 551--585.

Digital Library

[5]

Har-Peled, S., Roth, D., & Zimak, D. (2007). Maximum margin coresets for active and noise tolerant learning. Proceedings of International Joint Conference on Artificial Intelligence (pp. 836--841).

Digital Library

[6]

Joachims, T. (2006). Training linear SVMs in linear time. Proceedings of the International Conference on Knowledge Discovery and Data Mining (pp. 217--226).

Digital Library

[7]

Kumar, P., Mitchell, J., & Yildirim, A. (2003). Approximate minimum enclosing balls in high dimensions using coresets. ACM Journal of Experimental Algorithmics, 8.

Digital Library

[8]

Nock, R., & Nielsen, F. (2005). Fitting the smallest enclosing Bregman ball. Proceedings of the 16th European Conference on Machine Learning (pp. 649--656).

Digital Library

[9]

Osuna, E., Freund, R., & Girosi, F. (1997). An improved training algorithm for support vector machines. Proceedings of the IEEE Workshop on Neural Networks for Signal Processing (pp. 276--285).

[10]

Panigrahy, R. (2004). Minimum enclosing polytope in high dimensions. CoRR cs.CG/0407020.

[11]

Shevade, S., & Chu, W. (2006). Minimum enclosing spheres formulations for support vector ordinal regression. Proceedings of the Sixth International Conference on Data Mining (pp. 1054--1058).

Digital Library

[12]

Sonnenburg, S., Rätsch, G., Schäfer, C., & Schöölkopf, B. (2006). Large scale multiple kernel learning. Journal of Machine Learning Research, 7, 1531--1565.

Digital Library

[13]

Tsang, I. W., & Kwok, J. T. (2007). Large-scale sparsified manifold regularization. Advances in Neural Information Processing Systems 19. Cambridge, MA: MIT Press.

[14]

Tsang, I. W., Kwok, J. T., & Cheung, P.-M. (2005a). Core vector machines: Fast SVM training on very large data sets. Journal of Machine Learning Research, 6, 363--392.

Digital Library

[15]

Tsang, I. W., Kwok, J. T., & Lai, K. T. (2005b). Core vector regression for very large regression problems. Proceedings of the Twenty-Second International Conference on Machine Learning (pp. 913--920).

Digital Library

[16]

Vishwanathan, S., Smola, A., & Murty, M. (2003). SimpleSVM. Proceedings of the Twentieth International Conference on Machine Learning (pp. 760--767).

Cited By

Du KJiang BLu JHua JSwamy M(2024)Exploring Kernel Machines and Support Vector Machines: Principles, Techniques, and Future DirectionsMathematics10.3390/math1224393512:24(3935)Online publication date: 13-Dec-2024
https://doi.org/10.3390/math12243935
Nguyen VLe TTantithamthavorn CGrundy JPhung D(2024)Deep Domain Adaptation With Max-Margin Principle for Cross-Project Imbalanced Software Vulnerability DetectionACM Transactions on Software Engineering and Methodology10.1145/366460233:6(1-34)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3664602
Pimentel JOspina RAra A(2024)A novel fusion Support Vector Machine integrating weak and sphere models for classification challenges with massive dataDecision Analytics Journal10.1016/j.dajour.2024.10045711(100457)Online publication date: Jun-2024
https://doi.org/10.1016/j.dajour.2024.100457
Show More Cited By

Simpler core vector machines with enclosing balls
1. Computing methodologies

Recommendations

Generalized Core Vector Machines

Kernel methods, such as the support vector machine (SVM), are often formulated as quadratic programming (QP) problems. However, given m training patterns, a naive implementation of the QP solver takes O(m ³) training time and at least O(m²) space. Hence,...
Smaller core-sets for balls
SODA '03: Proceedings of the fourteenth annual ACM-SIAM symposium on Discrete algorithms

Given a set of points P ⊂ R^d and value ∊ > 0, an ∊-core-set S ⊂ P has the property that the smallest ball containing S is an ∊-approximation of the smallest ball containing P. This paper shows that any point-set has an ∊-core-set of size [2/∊]. We also ...
Approximate minimum enclosing balls in high dimensions using core-sets

We study the minimum enclosing ball (MEB) problem for sets of points or balls in high dimensions. Using techniques of second-order cone programming and "core-sets", we have developed (1 + ε)-approximation algorithms that perform well in practice, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '07: Proceedings of the 24th international conference on Machine learning

June 2007

1233 pages

ISBN:9781595937933

DOI:10.1145/1273496

Editor:
Zoubin Ghahramani
University of Cambridge, United Kingdom

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Machine Learning Journal

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 June 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

ICML '07 & ILP '07

Sponsor:

ICML '07 & ILP '07: The 24th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

June 20 - 24, 2007

Oregon, Corvalis, USA

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

139
Total Citations
View Citations
518
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)2

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Du KJiang BLu JHua JSwamy M(2024)Exploring Kernel Machines and Support Vector Machines: Principles, Techniques, and Future DirectionsMathematics10.3390/math1224393512:24(3935)Online publication date: 13-Dec-2024
https://doi.org/10.3390/math12243935
Nguyen VLe TTantithamthavorn CGrundy JPhung D(2024)Deep Domain Adaptation With Max-Margin Principle for Cross-Project Imbalanced Software Vulnerability DetectionACM Transactions on Software Engineering and Methodology10.1145/366460233:6(1-34)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3664602
Pimentel JOspina RAra A(2024)A novel fusion Support Vector Machine integrating weak and sphere models for classification challenges with massive dataDecision Analytics Journal10.1016/j.dajour.2024.10045711(100457)Online publication date: Jun-2024
https://doi.org/10.1016/j.dajour.2024.100457
Jennath HAsharaf S(2024)Scalable decision fusion algorithm for enabling decentralized computation in distributed, big data clustering problemsInternational Journal of Machine Learning and Cybernetics10.1007/s13042-024-02121-715:9(3803-3827)Online publication date: 8-Apr-2024
https://doi.org/10.1007/s13042-024-02121-7
Gottlieb LKaufman EKontorovich ANivasch GPele O(2024)Nested barycentric coordinate system as an explicit feature map for polyhedra approximation and learning tasksMachine Learning10.1007/s10994-024-06596-x113:10(7807-7840)Online publication date: 5-Sep-2024
https://doi.org/10.1007/s10994-024-06596-x
Lian SLian S(2024)Several Further Research Directions and TopicsPrinciples of Imprecise-Information Processing10.1007/978-981-97-6425-9_26(623-632)Online publication date: 3-Nov-2024
https://doi.org/10.1007/978-981-97-6425-9_26
Akhtar MQadri SSiddiqui MMustafa SJavaid SAli S(2023)Robust genetic machine learning ensemble model for intrusion detection in network trafficScientific Reports10.1038/s41598-023-43816-113:1Online publication date: 11-Oct-2023
https://doi.org/10.1038/s41598-023-43816-1
Afshin BShiri MLayeghi KHaj Seyyed Javadi H(2023)Kernel Optimization for Reducing Core Vector Machine Classification ErrorNeural Processing Letters10.1007/s11063-023-11236-x55:7(10011-10036)Online publication date: 1-Aug-2023
https://doi.org/10.1007/s11063-023-11236-x
Agarwal RAgarwal KHorsch APrasad D(2023)Auxiliary Network: Scalable and Agile Online Learning for Dynamic System with Inconsistently Available InputsNeural Information Processing10.1007/978-3-031-30105-6_46(549-561)Online publication date: 13-Apr-2023
https://doi.org/10.1007/978-3-031-30105-6_46
Clever LPohl JBossek JKerschke PTrautmann H(2022)Process-Oriented Stream Classification Pipeline: A Literature ReviewApplied Sciences10.3390/app1218909412:18(9094)Online publication date: 9-Sep-2022
https://doi.org/10.3390/app12189094
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten