skip to main content
10.1145/1273496.1273560acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicmlConference Proceedingsconference-collections
Article

Support cluster machine

Published: 20 June 2007 Publication History

Abstract

For large-scale classification problems, the training samples can be clustered beforehand as a downsampling pre-process, and then only the obtained clusters are used for training. Motivated by such assumption, we proposed a classification algorithm, Support Cluster Machine (SCM), within the learning framework introduced by Vapnik. For the SCM, a compatible kernel is adopted such that a similarity measure can be handled not only between clusters in the training phase but also between a cluster and a vector in the testing phase. We also proved that the SCM is a general extension of the SVM with the RBF kernel. The experimental results confirm that the SCM is very effective for largescale classification problems due to significantly reduced computational costs for both training and testing and comparable classification accuracies. As a by-product, it provides a promising approach to dealing with privacy-preserving data mining problems.

References

[1]
Boley, D., & Cao, D. (2004). Training support vector machine using adaptive clustering. Proc. of the SIAM Int'l Conf. on Data Mining.
[2]
Campbell, W. M., Sturim, D. E., & Reynolds, D. A. (2006). Support vector machines using GMM supervectors for speaker verification. IEEE Signal Processing Letters, 13, 308--311.
[3]
Cauwenberghs, G., & Poggio, T. (2000). Incremental and decremental support vector machine learning. Advanced Neural Information Processing Systems. Cambridge, MA: MIT Press.
[4]
Chang, C., & Lin, C. (2001). LIBSVM: A library for support vector machines.
[5]
Collobert, R., & Bengio, S. (2001). SVMTorch: Support vector machines for large-scale regression problems. J. of Machine Learning Research, 1, 143--160.
[6]
Collobert, R., Bengio, S., & Bengio, Y. (2001). A parallel mixture of SVMs for very large scale problems. Advanced Neural Information Processing Systems. Cambridge, MA: MIT Press.
[7]
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 39, 1--38.
[8]
Friedman, M., & Kandel, A. (1999). Introduction to pattern recognition, chapter Distance Functions, 70--73. London, UK: Imperial College Press.
[9]
Fung, G., & Mangasarian, O. L. (2001). Proximal support vector machine classifiers. Proc. of the ACM SIGKDD Int'l Conf. on Knowledge Discovery and Data Mining (pp. 77--86).
[10]
Fung, G., & Mangasarian, O. L. (2002). Incremental support vector machine classification. Proc. of the SIAM Int'l Conf. on Data Mining.
[11]
Graf, H. P., Cosatto, E., Bottou, L., Durdanovic, I., & Vapnik, V. (2004). Parallel support vector machines: The cascade SVM. Advanced Neural Information Processing Systems. Cambridge, MA: MIT Press.
[12]
Hartigan, J. A., & Wong, M. A. (1979). A K-Means clustering algorithm. Applied Statistics, 28, 100--108.
[13]
Jebara, T., Kondor, R., & Howard, A. (2004). Probability product kernels. J. of Machine Learning Research, 5, 819--844.
[14]
Joachims, T. (1999). Making large-scale svm learning practical. In B. Schölkopf, C. J. C. Burges and A. J. Smola (Eds.), Advances in kernel methods - support vector learning, 169--184. Cambridge, MA: MIT Press.
[15]
Keerthi, S. S., Shevade, S. K., Bhattacharyya, C., & Murthy, K. R. K. (2001). Improvements to Platt's SMO algorithm for SVM classifier design. Neural Computation, 13, 637--649.
[16]
Laskov, P., Gehl, C., Krüger, S., & Müüller, K. (2006). Incremental support vector learning: Analysis, implementation and applications. J. of Machine Learning Research, 7, 1909--1936.
[17]
Lee, Y., & Mangasarian, O. L. (2001). RSVM: Reduced support vector machines. Proc. of the SIAM Int'l Conf. on Data Mining.
[18]
Moreno, P. J., Ho, P. P., & Vasconcelos, N. (2003). A Kullback-Leibler divergence based kernel for SVM classification in multimedia applications. Advanced Neural Information Processing Systems. Cambridge, MA: MIT Press.
[19]
Osuna, E., Freund, R., & Girosi, F. (1997). An improved training algorithm for support vector machines. Proc. of the 1997 IEEE Workshop on Neural Networks for Signal Processing (pp. 276--285).
[20]
Platt, J. (1999). Fast training of support vector machines using sequential minimal optimization. In B. Schölkopf, C. J. C. Burges and A. J. Smola (Eds.), Advances in kernel methods - support vector learning, 185--208. Cambridge, MA: MIT Press.
[21]
Schohn, G., & Cohn, D. (2000). Less is More: Active learning with support vector machines. Proc. of the Int'l Conf. on Machine Learning.
[22]
Sun, S., Tseng, C. L., Chen, Y. H., Chuang, S. C., & Fu, H. C. (2004). Cluster-based support vector machines in text-independent speaker identification. Proc. of the Int'l Joint Conf. on Neural Network.
[23]
Tsang, I. W., Kwok, J. T., & Cheung, P. (2005). Core Vector Machines: Fast SVM training on very large data sets. J. of Machine Learning Research, 6, 363--392.
[24]
Vapnik, V. (1998). Statistical learning theory. John Wiley.
[25]
Yu, H., Yang, J., & Han, J. (2003). Classifying large data sets using SVMs with hierarchical clusters. Proc. of the ACM SIGKDD Int'l Conf. on Knowledge Discovery and Data Mining (pp. 306--315).
[26]
Yuan, J., Li, J., & Zhang, B. (2006). Learning concepts from large scale imbalanced data sets using support cluster machines. Proc. of the ACM Int'l Conf. on Multimedia (pp. 441--450).
[27]
Zhang, T., Ramakrishnan, R., & Livny, M. (1996). BIRCH: An efficient data clustering method for very large databases. Proc. of the 1996 ACM SIGMOD Int'l Conf. on Management of Data (pp. 103--114).

Cited By

View all
  • (2020)Classifying With Adaptive Hyper-Spheres: An Incremental Classifier Based on Competitive LearningIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2017.276136050:4(1218-1229)Online publication date: Apr-2020
  • (2017)Remote Sensing Image Classification: A survey of support-vector-machine-based advanced techniquesIEEE Geoscience and Remote Sensing Magazine10.1109/MGRS.2016.26412405:1(33-52)Online publication date: Mar-2017
  • (2015)L3-SVM: a lifelong learning method for SVM2015 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2015.7280379(1-8)Online publication date: Jul-2015
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICML '07: Proceedings of the 24th international conference on Machine learning
June 2007
1233 pages
ISBN:9781595937933
DOI:10.1145/1273496
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

  • Machine Learning Journal

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 June 2007

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Article

Conference

ICML '07 & ILP '07
Sponsor:

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)3
  • Downloads (Last 6 weeks)0
Reflects downloads up to 24 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2020)Classifying With Adaptive Hyper-Spheres: An Incremental Classifier Based on Competitive LearningIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2017.276136050:4(1218-1229)Online publication date: Apr-2020
  • (2017)Remote Sensing Image Classification: A survey of support-vector-machine-based advanced techniquesIEEE Geoscience and Remote Sensing Magazine10.1109/MGRS.2016.26412405:1(33-52)Online publication date: Mar-2017
  • (2015)L3-SVM: a lifelong learning method for SVM2015 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2015.7280379(1-8)Online publication date: Jul-2015
  • (2015)The Instance and Feature Selection for Neural Network Based Diagnosis of Chronic Obstructive BronchitisApplications of Computational Intelligence in Biomedical Technology10.1007/978-3-319-19147-8_13(215-228)Online publication date: 26-Jun-2015
  • (2014)Incremental Learning with Support Vector Data DescriptionProceedings of the 2014 22nd International Conference on Pattern Recognition10.1109/ICPR.2014.669(3904-3909)Online publication date: 24-Aug-2014
  • (2014)Clustering for Data Privacy and Classification TasksOperations Research Proceedings 201310.1007/978-3-319-07001-8_54(397-403)Online publication date: 10-Jul-2014
  • (2013)Particle swarm optimization for support vector clustering Separating hyper-plane of unlabeled data2013 5th International Conference on Modeling, Simulation and Applied Optimization (ICMSAO)10.1109/ICMSAO.2013.6552696(1-6)Online publication date: Apr-2013
  • (2013)Symbolic Cluster Representations for SVM in Credit Client Classification TasksStatistical Models for Data Analysis10.1007/978-3-319-00032-9_40(353-360)Online publication date: 22-May-2013
  • (2012)An online incremental learning support vector machine for large-scale dataNeural Computing and Applications10.1007/s00521-011-0793-122:5(1023-1035)Online publication date: 5-Jan-2012
  • (2012)Clustering and categorization of Brazilian portuguese legal documentsProceedings of the 10th international conference on Computational Processing of the Portuguese Language10.1007/978-3-642-28885-2_31(272-283)Online publication date: 17-Apr-2012
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media