poster

A GPU-tailored approach for training kernelized SVMs

Authors:

Andrew Cotter,

Nathan Srebro,

Joseph KeshetAuthors Info & Claims

KDD '11: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 805 - 813

https://doi.org/10.1145/2020408.2020548

Published: 21 August 2011 Publication History

Get Access

Abstract

We present a method for efficiently training binary and multiclass kernelized SVMs on a Graphics Processing Unit (GPU). Our methods apply to a broad range of kernels, including the popular Gaus- sian kernel, on datasets as large as the amount of available memory on the graphics card. Our approach is distinguished from earlier work in that it cleanly and efficiently handles sparse datasets through the use of a novel clustering technique. Our optimization algorithm is also specifically designed to take advantage of the graphics hardware. This leads to different algorithmic choices then those preferred in serial implementations. Our easy-to-use library is orders of magnitude faster then existing CPU libraries, and several times faster than prior GPU approaches.

References

[1]

A. Bordes, S. Ertekin, J. Weston, and L. Bottou. Fast kernel classifiers with online and active learning. JMLR, 6: 1579--1619, September 2005.

Digital Library

Google Scholar

[2]

A. Carpenter. CUSVM: A CUDA implementation of support vector classification and regression. http://patternsonascreen.net/cuSVM.html, 2009.

Google Scholar

[3]

B. Catanzaro, N. Sundaram, and K. Keutzer. Fast support vector machine training and classification on graphics processors. In ICML'08, pages 104--111, 2008.

Digital Library

Google Scholar

[4]

C.-C. Chang and C.-J. Lin. phLIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/ cjlin/libsvm.

Digital Library

Google Scholar

[5]

K. Crammer and Y. Singer. On the algorithmic implementation of multiclass kernel-based vector machines. phJMLR, 2: 265--292, March 2002. ISSN 1532--4435.

Digital Library

Google Scholar

[6]

T.-N. Do, V.-H. Nguyen, and F. Poulet. Speed up SVM algorithm for massive classification tasks. In ADMA'08, pages 147--157, Berlin, Heidelberg, 2008. Springer-Verlag.

Digital Library

Google Scholar

[7]

R.-E. Fan, P.-S. Chen, and C.-J. Lin. Working set selection using second order information for training support vector machines. JMLR, 6: 1889--1918, 2005.

Digital Library

Google Scholar

[8]

C.-J. Hsieh, K.-W. Chang, C.-J. Lin, S. S. Keerthi, and S. Sundararajan. A dual coordinate descent method for large-scale linear SVM. In ICML'08, pages 408--415, 2008.

Digital Library

Google Scholar

[9]

T. Joachims. Making large-scale support vector machine learning practical. In B. Schölkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods - Support Vector Learning. MIT Press, 1998.

Digital Library

Google Scholar

[10]

E. Osuna, R. Freund, and F. Girosi. Training support vector machines: an application to face detection. In CVPR'97, June 1997.

Digital Library

Google Scholar

[11]

J. C. Platt. Fast training of support vector machines using Sequential Minimal Optimization. In B. Schölkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods - Support Vector Learning. MIT Press, 1998.

Digital Library

Google Scholar

[12]

B. Schölkopf and A. J. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge, MA, USA, 2001. ISBN 0262194759.

Digital Library

Google Scholar

[13]

S. Shalev-Shwartz, Y. Singer, and N. Srebro. Pegasos: Primal Estimated sub-GrAdient SOlver for SVM. In ICML'07, pages 807--814, 2007.

Digital Library

Google Scholar

[14]

S. Shalev-Shwartz, Y. Singer, N. Srebro, and A. Cotter. Pegasos: Primal Estimated sub-GrAdient SOlver for SVM. Mathematical Programming, pages 1--34, October 2010.

Digital Library

Google Scholar

Cited By

View all

Chen YWen ZHe BChen J(2023)Efficient Decomposition Selection for Multi-class ClassificationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.313023935:4(3751-3764)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TKDE.2021.3130239
Xu BWen ZYan LZhao ZYin ZLiu WHe B(2023)Leveraging Data Density and Sparsity for Efficient SVM Training on GPUs2023 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM58522.2023.00079(698-707)Online publication date: 1-Dec-2023
https://doi.org/10.1109/ICDM58522.2023.00079
Van Craen ABreyer MPflüger D(2022)PLSSVM—Parallel Least Squares Support Vector MachineSoftware Impacts10.1016/j.simpa.2022.10034314(100343)Online publication date: Nov-2022
https://doi.org/10.1016/j.simpa.2022.100343
Show More Cited By

Index Terms

A GPU-tailored approach for training kernelized SVMs
1. Computing methodologies
  1. Concurrent computing methodologies
    1. Concurrent programming languages
2. Software and its engineering
  1. Software notations and tools
    1. General programming languages
      1. Language types
        Concurrent programming languages

Recommendations

On the Efficacy of a Fused CPU+GPU Processor (or APU) for Parallel Computing
SAAHPC '11: Proceedings of the 2011 Symposium on Application Accelerators in High-Performance Computing

The graphics processing unit (GPU) has made significant strides as an accelerator in parallel computing. However, because the GPU has resided out on PCIe as a discrete device, the performance of GPU applications can be bottlenecked by data transfers ...
Optimized HPL for AMD GPU and multi-core CPU usage

The installation of the LOEWE-CSC ( http://csc.uni-frankfurt.de/csc/__ __51 ) supercomputer at the Goethe University in Frankfurt lead to the development of a Linpack which can fully utilize the installed AMD Cypress GPUs. At its core, a fast DGEMM for ...
Accelerating the discontinuous Galerkin method for seismic wave propagation simulations using the graphic processing unit (GPU)-single-GPU implementation

We have successfully ported an arbitrary high-order discontinuous Galerkin (ADER-DG) method for solving the three-dimensional elastic seismic wave equation on unstructured tetrahedral meshes to an Nvidia Tesla C2075 GPU using the Nvidia CUDA programming ...

Comments

Information & Contributors

Information

Published In

KDD '11: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining

August 2011

1446 pages

ISBN:9781450308137

DOI:10.1145/2020408

General Chair:
Chid Apte
IBM Research
,
Program Chairs:
Joydeep Ghosh
UT Austin
,
Padhraic Smyth
UC Irvine

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 August 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tag

GPGPU

Qualifiers

Poster

Conference

KDD '11

Sponsor:

KDD '11: The 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 21 - 24, 2011

California, San Diego, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
371
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)2

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Chen YWen ZHe BChen J(2023)Efficient Decomposition Selection for Multi-class ClassificationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.313023935:4(3751-3764)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TKDE.2021.3130239
Xu BWen ZYan LZhao ZYin ZLiu WHe B(2023)Leveraging Data Density and Sparsity for Efficient SVM Training on GPUs2023 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM58522.2023.00079(698-707)Online publication date: 1-Dec-2023
https://doi.org/10.1109/ICDM58522.2023.00079
Van Craen ABreyer MPflüger D(2022)PLSSVM—Parallel Least Squares Support Vector MachineSoftware Impacts10.1016/j.simpa.2022.10034314(100343)Online publication date: Nov-2022
https://doi.org/10.1016/j.simpa.2022.100343
Bhajantri VTotad SBharamagoudar G(2022)Enhancement of Scalability of SVM Classifiers for Big DataAdvances in Data Science and Analytics10.1002/9781119792826.ch9(203-232)Online publication date: 31-Oct-2022
https://doi.org/10.1002/9781119792826.ch9
Diaz-Morales RNavia-Vazquez A(2020)Distributed Nonlinear Semiparametric Support Vector Machine for Big Data Applications on Spark FrameworksIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2018.285877850:11(4664-4675)Online publication date: Nov-2020
https://doi.org/10.1109/TSMC.2018.2858778
Tavara S(2019)Parallel Computing of Support Vector MachinesACM Computing Surveys10.1145/328098951:6(1-38)Online publication date: 28-Jan-2019
https://dl.acm.org/doi/10.1145/3280989
Li QWen ZHe B(2019)Adaptive Kernel Value Caching for SVM TrainingIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2019.2944562(1-11)Online publication date: 2019
https://doi.org/10.1109/TNNLS.2019.2944562
Wen ZShi JHe BChen JChen Y(2019)Efficient Multi-Class Probabilistic SVMs on GPUsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2018.286609731:9(1693-1706)Online publication date: 1-Sep-2019
https://doi.org/10.1109/TKDE.2018.2866097
Peng LZhang HHassan HChen YYang B(2019)Accelerating data gravitation-based classification using GPUThe Journal of Supercomputing10.1007/s11227-018-2253-575:6(2930-2949)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s11227-018-2253-5
Wen ZZhang RRamamohanarao KYang L(2018)Scalable and fast SVM regression using modern hardwareWorld Wide Web10.1007/s11280-017-0445-121:2(261-287)Online publication date: 1-Mar-2018
https://dl.acm.org/doi/10.1007/s11280-017-0445-1
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

On the Efficacy of a Fused CPU+GPU Processor (or APU) for Parallel Computing

Optimized HPL for AMD GPU and multi-core CPU usage

Accelerating the discontinuous Galerkin method for seismic wave propagation simulations using the graphic processing unit (GPU)-single-GPU implementation

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tag

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations