skip to main content
10.1145/2638404.2638474acmotherconferencesArticle/Chapter ViewAbstractPublication Pagesacm-seConference Proceedingsconference-collections
research-article

A survey of GPU accelerated SVM

Published: 28 March 2014 Publication History

Abstract

Support Vector Machines (SVM) is a set of machine learning algorithms that have been widely used in diverse domains. As the volume of data generated by humans and machines increases year by year, the traditional training algorithms for SVM become infeasible for large scale datasets. Mathematical optimization approaches and computing parallel techniques are two popular strategies to accelerate the training process of SVM. Among those parallel approaches, implementing SVM on Graphics Processing Units (GPUs) has become new research and application interest. General used GPUs have been widely adopted to accelerate a lot of traditional algorithms, including SVM and achieved high performance and speedup. In this work, we survey the mathematical optimization algorithms of SVM training process, as well as GPU accelerated implementations of SVM.

References

[1]
A. Athanasopoulos, A. D., V. Mezaris, I. Kompatsiaris 2011. GPU Acceleration for Support Vector Machines. In 12th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2011), Delft, The Netherlands.
[2]
Blanz, V., Schölkopf, B., Bülthoff, H., Burges, C., Vapnik, V., and Vetter, T., 1996. Comparison of view-based object recognition algorithms using realistic 3D models. In Artificial Neural Networks -- ICANN 96, C. Malsburg, W. Seelen, J. Vorbrüggen and B. Sendhoff Eds. Springer Berlin Heidelberg, 251--256. DOI= http://dx.doi.org/10.1007/3-540-61510-5_45.
[3]
Burges, C. J. C., 1998. A tutorial on Support Vector Machines for pattern recognition. Data Mining and Knowledge Discovery 2, 2 (Jun), 121--167. DOI= http://dx.doi.org/Doi 10.1023/A:1009715923555.
[4]
Burges, C. J. C. and Scholkopf, B., 1997. Improving the Accuracy and Speed of Support Vector Machines. In Advances in Neural Information Processing Systems 9 MIT Press, 375--381.
[5]
Carpenter, A., 2009. CUSVM: A Cuda Implementation of Support Vector Classification and Regression. http://patternsonascreen.net/cuSVM.html.
[6]
Catanzaro, B., Sundaram, N., and Keutzer, K., 2008. Fast support vector machine training and classification on graphics processors. In Proceedings of the Proceedings of the 25th international conference on Machine learning (Helsinki, Finland2008), ACM, 1390170, 104--111. DOI= http://dx.doi.org/10.1145/1390156.1390170.
[7]
Cortes, C. and Vapnik, V., 1995. Support-Vector Networks. Machine Learning 20, 3 (Sep), 273--297. DOI= http://dx.doi.org/Doi 10.1023/A:1022627411411.
[8]
Fan, R.-E., Chen, P.-H., and Lin, C.-J., 2005. Working Set Selection Using Second Order Information for Training Support Vector Machines. J. Mach. Learn. Res. 6, 1889--1918.
[9]
Friedman, J. H., 1996. Another approach to polychotomous classification. Department of Statistics, Stanford University.
[10]
Herrero-Lopez, S., Williams, J. R., and Sanchez, A., 2010. Parallel multiclass classification using SVMs on GPUs. In Proceedings of the Proceedings of the 3rd Workshop on General-Purpose Computation on Graphics Processing Units (Pittsburgh, Pennsylvania2010), ACM, 1735692, 2--11. DOI= http://dx.doi.org/10.1145/1735688.1735692.
[11]
Joachims, T., 1998. Text categorization with Support Vector Machines: Learning with many relevant features. In Machine Learning: ECML-98, C. Nédellec and C. Rouveirol Eds. Springer Berlin Heidelberg, 137--142. DOI= http://dx.doi.org/10.1007/BFb0026683.
[12]
Joachims, T., 1999. Making large-scale support vector machine learning practical. In Advances in kernel methods MIT Press, 169--184.
[13]
Kaufman, L., 1999. Solving the quadratic programming problem arising in support vector classification. In Advances in kernel methods MIT Press, 147--167.
[14]
Keerthi, S. S., Shevade, S. K., Bhattacharyya, C., and Murthy, K. R. K., 2001. Improvements to Platt's SMO Algorithm for SVM Classifier Design. Neural Computation 13, 3 (2001/03/01), 637--649. DOI= http://dx.doi.org/10.1162/089976601300014493.
[15]
Li, Q., Salman, R., and Kecman, V., 2010. An intelligent system for accelerating parallel SVM classification problems on large datasets using GPU. In Intelligent Systems Design and Applications (ISDA), 2010 10th International Conference on, 1131--1135. DOI= http://dx.doi.org/10.1109/isda.2010.5687033.
[16]
Li, Q., Salman, R., Test, E., Strack, R., and Kecman, V., 2013. Parallel multitask cross validation for Support Vector Machine using GPU. Journal of Parallel and Distributed Computing 73, 3 (Mar), 293--302. DOI= http://dx.doi.org/DOI 10.1016/j.jpdc.2012.02.011.
[17]
Liao, Q., Wang, J. B., Webster, Y., and Watson, I. A., 2009. GPU Accelerated Support Vector Machines for Mining High-Throughput Screening Data. Journal of chemical information and modeling 49, 12 (Dec), 2718--2725. DOI= http://dx.doi.org/Doi 10.1021/Ci900337f.
[18]
LIBSVM, 2013. http://www.csie.ntu.edu.tw/~cjlin/libsvm/
[19]
Lin, T. K. and Chien, S. Y., 2010. Support Vector Machines on GPU with Sparse Matrix Format. In Machine Learning and Applications (ICMLA), 2010 Ninth International Conference on, 313--318. DOI= http://dx.doi.org/10.1109/icmla.2010.53.
[20]
Mangasarian, O. L. and Musicant, D. R., 1999. Successive overrelaxation for support vector machines. Neural Networks, IEEE Transactions on 10, 5, 1032--1037. DOI= http://dx.doi.org/10.1109/72.788643.
[21]
Nickolls, J., Buck, I., Garland, M., and Skadron, K., 2008. Scalable Parallel Programming with CUDA. Queue 6, 2, 40--53. DOI= http://dx.doi.org/10.1145/1365490.1365500.
[22]
Osuna, E., Freund, R., and Girosi, F., 1997. An improved training algorithm for support vector machines. In Neural Networks for Signal Processing {1997} VII. Proceedings of the 1997 IEEE Workshop, 276--285. DOI= http://dx.doi.org/10.1109/nnsp.1997.622408.
[23]
Osuna, E., Freund, R., and Girosi, F., 1997. Training support vector machines: an application to face detection. In Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 IEEE Computer Society Conference on, 130--136. DOI= http://dx.doi.org/10.1109/cvpr.1997.609310.
[24]
Peng, N. B., Zhang, Y. X., and Zhao, Y. H., 2011. CUDA-Accelerated SVM for Celestial Object Classification. Astronomical Data Analysis Software and Systems Xx 442, 119--122.
[25]
Platt, J. C., 1999. Fast training of support vector machines using sequential minimal optimization. In Advances in kernel methods MIT Press, 185--208.
[26]
Saunders, R., Stitson, C., Weston, J., Bottou, L., Scholkopf, B., and Smola, A., 1998. Support Vector Machine reference manual.
[27]
Schmidt, M. and Gish, H., 1996. Speaker identification via support vector classifiers. In Proceedings of the Proceedings of the Acoustics, Speech, and Signal Processing, 1996. on Conference Proceedings., 1996 IEEE International Conference - Volume 01 (1996), IEEE Computer Society, 1256446, 105--108. DOI= http://dx.doi.org/10.1109/icassp.1996.540301.
[28]
Scholkopf, B., Burges, C., and Vapnik, V., 1996. Incorporating Invariances in Support Vector Learning Machines Springer, 47--52.
[29]
Sopy, K., #322, Pawe, Drozda, Przemys, G, A., #243, and Recki, 2012. SVM with CUDA accelerated kernels for big sparse problems. In Proceedings of the Proceedings of the 11th international conference on Artificial Intelligence and Soft Computing - Volume Part I (Zakopane, Poland2012), Springer-Verlag, 2342010, 439--447. DOI= http://dx.doi.org/10.1007/978-3-642-29347-4_51.
[30]
Vapnik, V. N., 1982. Estimation of Dependences Based on Empirical Data. Springer Verlag, New York.
[31]
Vapnik, V. N., 1995 The Nature of Statistical Learning Theory. Springer-Verlag, New York.
[32]
Weimer, D., Kohler, S., Hellert, C., Doll, K., Brunsmann, U., and Krzikalla, R., 2011. Gpu architecture for stationary multisensor pedestrian detection at smart intersections. In Intelligent Vehicles Symposium (IV), 2011 IEEE, 89--94. DOI= http://dx.doi.org/10.1109/ivs.2011.5940411.
[33]
Yugov., V. and Kumazawa., I., 2013. Combination of SVM and FERN for GPU-assisted texture recognition on mobile devices ACSIJ Advances in Computer Science: an International Journal 2, 3 (July).
[34]
Zoutendijk, G., 1970. Nonlinear Programming, Computational Methods. In J. Abadie Ed, Integer and Nonlinear Programming, Amsterdam: North-Holland, 37--86.

Cited By

View all
  • (2022)Resource Scheduling Strategy for Spark in Co-allocated Data CentersProceeding of 2021 International Conference on Wireless Communications, Networking and Applications10.1007/978-981-19-2456-9_13(114-122)Online publication date: 13-Jul-2022
  • (2022)GPU Accelerated Modelling and Forecasting for Large Time SeriesComputational Science – ICCS 202210.1007/978-3-031-08757-8_33(398-412)Online publication date: 21-Jun-2022
  • (2022)Enhancement of Scalability of SVM Classifiers for Big DataAdvances in Data Science and Analytics10.1002/9781119792826.ch9(203-232)Online publication date: 31-Oct-2022
  • Show More Cited By

Index Terms

  1. A survey of GPU accelerated SVM

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ACMSE '14: Proceedings of the 2014 ACM Southeast Conference
    March 2014
    265 pages
    ISBN:9781450329231
    DOI:10.1145/2638404
    • Conference Chair:
    • Ken Hoganson,
    • Program Chair:
    • Selena He
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 28 March 2014

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. CUDA
    2. GPU
    3. SVM
    4. optimization

    Qualifiers

    • Research-article

    Conference

    ACM SE '14
    ACM SE '14: ACM Southeast Regional Conference 2014
    March 28 - 29, 2014
    Georgia, Kennesaw

    Acceptance Rates

    Overall Acceptance Rate 502 of 1,023 submissions, 49%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)11
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 30 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)Resource Scheduling Strategy for Spark in Co-allocated Data CentersProceeding of 2021 International Conference on Wireless Communications, Networking and Applications10.1007/978-981-19-2456-9_13(114-122)Online publication date: 13-Jul-2022
    • (2022)GPU Accelerated Modelling and Forecasting for Large Time SeriesComputational Science – ICCS 202210.1007/978-3-031-08757-8_33(398-412)Online publication date: 21-Jun-2022
    • (2022)Enhancement of Scalability of SVM Classifiers for Big DataAdvances in Data Science and Analytics10.1002/9781119792826.ch9(203-232)Online publication date: 31-Oct-2022
    • (2020)Distributed Nonlinear Semiparametric Support Vector Machine for Big Data Applications on Spark FrameworksIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2018.285877850:11(4664-4675)Online publication date: Nov-2020
    • (2019)Parallel Computing of Support Vector MachinesACM Computing Surveys10.1145/328098951:6(1-38)Online publication date: 28-Jan-2019
    • (2017)A survey on graphic processing unit computing for large‐scale data miningWIREs Data Mining and Knowledge Discovery10.1002/widm.12328:1Online publication date: 7-Nov-2017
    • (2016)Accelerating SVM with GPU: The State of the ArtArtificial Intelligence and Soft Computing10.1007/978-3-319-39384-1_55(624-634)Online publication date: 29-May-2016

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media