Skip to main content
SpringerLink
Log in

Non-linear Classification of Massive Datasets with a Parallel Algorithm of Local Support Vector Machines

  • Conference paper
Advanced Computational Methods for Knowledge Engineering

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 358))

Abstract

We propose a new parallel algorithm of local support vector machines, called kSVM for the effectively non-linear classification of large datasets. The learning strategy of kSVM uses kmeans algorithm to partition the data into k clusters, followed which it constructs a non-linear SVM in each cluster to classify the data locally in the parallel way on multi-core computers. The kSVM algorithm is faster than the standard SVM in the non-linear classification of large datasets while maintaining the classification correstness. The numerical test results on 4 datasets from UCI repository and 3 benchmarks of handwritten letters recognition showed that our proposal is efficient compared to the standard SVM.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Vapnik, V.: The Nature of Statistical Learning Theory. Springer (1995)

    Google Scholar 

  2. Guyon, I.: Web page on svm applications (1999), http://www.clopinet.com/isabelle/Projects/-SVM/app-list.html

  3. MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 281–297. University of California Press, Berkeley (1967)

    Google Scholar 

  4. Asuncion, A., Newman, D.: UCI repository of machine learning databases (2007)

    Google Scholar 

  5. LeCun, Y., Boser, B., Denker, J., Henderson, D., Howard, R., Hubbard, W., Jackel, L.: Backpropagation applied to handwritten zip code recognition. Neural Computation 1(4), 541–551 (1989)

    Article  Google Scholar 

  6. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  7. van der Maaten, L.: A new benchmark dataset for handwritten character recognition (2009), http://homepage.tudelft.nl/19j49/Publications_files/characters.zip

  8. Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines: And Other Kernel-based Learning Methods. Cambridge University Press, New York (2000)

    Book  MATH  Google Scholar 

  9. Platt, J.: Fast training of support vector machines using sequential minimal optimization. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods – Support Vector Learning, pp. 185–208 (1999)

    Google Scholar 

  10. Vapnik, V.: Principles of risk minimization for learning theory. In: Advances in Neural Information Processing Systems 4 [NIPS Conference, Denver, Colorado, USA, December 2-5, 1991], pp. 831–838 (1991)

    Google Scholar 

  11. Bottou, L., Vapnik, V.: Local learning algorithms. Neural Computation 4(6), 888–900 (1992)

    Article  Google Scholar 

  12. Vapnik, V., Bottou, L.: Local algorithms for pattern recognition and dependencies estimation. Neural Computation 5(6), 893–909 (1993)

    Article  Google Scholar 

  13. OpenMP Architecture Review Board: OpenMP application program interface version 3.0 (2008)

    Google Scholar 

  14. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2(27), 1–27 (2011)

    Article  Google Scholar 

  15. Lin, C.: A practical guide to support vector classification (2003)

    Google Scholar 

  16. Yu, H., Yang, J., Han, J.: Classifying large data sets using SVMs with hierarchical clusters. In: Proc. of the ACM SIGKDD Intl. Conf. on KDD, pp. 306–315. ACM (2003)

    Google Scholar 

  17. Do, T.N., Poulet, F.: Towards high dimensional data mining with boosting of psvm and visualization tools. In: Proc. of 6th Intl. Conf. on Entreprise Information Systems, pp. 36–41 (2004)

    Google Scholar 

  18. Boser, B., Guyon, I., Vapnik, V.: An training algorithm for optimal margin classifiers. In: Proc. of 5th ACM Annual Workshop on Computational Learning Theory, pp. 144–152. ACM (1992)

    Google Scholar 

  19. Osuna, E., Freund, R., Girosi, F.: An improved training algorithm for support vector machines. In: Principe, J., Gile, L., Morgan, N., Wilson, E. (eds.) Neural Networks for Signal Processing VII, pp. 276–285 (1997)

    Google Scholar 

  20. Mangasarian, O., Musicant, D.: Lagrangian support vector machines. Journal of Machine Learning Research 1, 161–177 (2001)

    MathSciNet  MATH  Google Scholar 

  21. Fung, G., Mangasarian, O.: Proximal support vector classifiers. In: Proc. of the ACM SIGKDD Intl. Conf. on KDD, pp. 77–86. ACM (2001)

    Google Scholar 

  22. Mangasarian, O.: A finite newton method for classification problems. Technical Report 01-11, Data Mining Institute, Computer Sciences Department, University of Wisconsin (2001)

    Google Scholar 

  23. Suykens, J., Vandewalle, J.: Least squares support vector machines classifiers. Neural Processing Letters 9(3), 293–300 (1999)

    Article  MATH  Google Scholar 

  24. Shalev-Shwartz, S., Singer, Y., Srebro, N.: Pegasos: Primal estimated sub-gradient solver for svm. In: Proceedings of the Twenty-Fourth International Conference ON Machine Learning, pp. 807–814. ACM (2007)

    Google Scholar 

  25. Bottou, L., Bousquet, O.: The tradeoffs of large scale learning. In: Platt, J., Koller, D., Singer, Y., Roweis, S. (eds.) Advances in Neural Information Processing Systems, vol. 20, pp. 161–168 (2008), http://books.nips.cc

  26. Do, T.N., Poulet, F.: Incremental svm and visualization tools for bio-medical data mining. In: Proc. of Workshop on Data Mining and Text Mining in Bioinformatics, pp. 14–19 (2003)

    Google Scholar 

  27. Do, T.N., Poulet, F.: Classifying one billion data with a new distributed svm algorithm. In: Proc. of 4th IEEE Intl. Conf. on Computer Science, Research, Innovation and Vision for the Future, pp. 59–66. IEEE Press (2006)

    Google Scholar 

  28. Fung, G., Mangasarian, O.: Incremental support vector machine classification. In: Proc. of the 2nd SIAM Int. Conf. on Data Mining (2002)

    Google Scholar 

  29. Poulet, F., Do, T.N.: Mining very large datasets with support vector machine algorithms. In: Camp, O., Filipe, J., Hammoudi, S., Piattini, M. (eds.) Enterprise Information Systems V, pp. 177–184 (2004)

    Google Scholar 

  30. Do, T.: Parallel multiclass stochastic gradient descent algorithms for classifying million images with very-high-dimensional signatures into thousands classes. Vietnam J. Computer Science 1(2), 107–115 (2014)

    Article  Google Scholar 

  31. Do, T.-N., Nguyen, V.-H., Poulet, F.: Speed up SVM algorithm for massive classification tasks. In: Tang, C., Ling, C.X., Zhou, X., Cercone, N.J., Li, X. (eds.) ADMA 2008. LNCS (LNAI), vol. 5139, pp. 147–157. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  32. Yu, H., Yang, J., Han, J.: Classifying large data sets using svms with hierarchical clusters. In: Proc. of the ACM SIGKDD Intl. Conf. on KDD, pp. 306–315. ACM (2003)

    Google Scholar 

  33. Do, T.N., Poulet, F.: Mining very large datasets with svm and visualization. In: Proc. of 7th Intl. Conf. on Entreprise Information Systems, pp. 127–134 (2005)

    Google Scholar 

  34. Boley, D., Cao, D.: Training support vector machines using adaptive clustering. In: Berry, M.W., Dayal, U., Kamath, C., Skillicorn, D.B. (eds.) Proceedings of the Fourth SIAM International Conference on Data Mining, Lake Buena Vista, Florida, USA, April 22-24, pp. 126–137. SIAM (2004)

    Google Scholar 

  35. Tong, S., Koller, D.: Support vector machine active learning with applications to text classification. In: Proc. of the 17th Intl. Conf. on Machine Learning, pp. 999–1006. ACM (2000)

    Google Scholar 

  36. Pavlov, D., Mao, J., Dom, B.: Scaling-up support vector machines using boosting algorithm. In: 15th International Conference on Pattern Recognition, vol. 2, pp. 219–222 (2000)

    Google Scholar 

  37. Do, T.N., Le-Thi, H.A.: Classifying large datasets with svm. In: Proc. of 4th Intl. Conf. on Computational Management Science (2007)

    Google Scholar 

  38. Do, T.N., Fekete, J.D.: Large scale classification with support vector machine algorithms. In: Wani, M.A., Kantardzic, M.M., Li, T., Liu, Y., Kurgan, L.A., Ye, J., Ogihara, M., Sagiroglu, S., Chen, X.W., Peterson, L.E., Hafeez, K. (eds.) The Sixth International Conference on Machine Learning and Applications, ICMLA 2007, Cincinnati, Ohio, USA, December 13-15, pp. 7–12. IEEE Computer Society (2007)

    Google Scholar 

  39. Freund, Y., Schapire, R.: A short introduction to boosting. Journal of Japanese Society for Artificial Intelligence 14(5), 771–780 (1999)

    Google Scholar 

  40. Breiman, L.: Arcing classifiers. The Annals of Statistics 26(3), 801–849 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  41. Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Computation 3(1), 79–87 (1991)

    Article  Google Scholar 

  42. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society, Series B 39(1), 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  43. Vincent, P., Bengio, Y.: K-local hyperplane and convex distance nearest neighbor algorithms. In: Advances in Neural Information Processing Systems, pp. 985–992. The MIT Press (2001)

    Google Scholar 

  44. Zhang, H., Berg, A., Maire, M., Malik, J.: SVM-KNN: Discriminative nearest neighbor classification for visual category recognition. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2126–2136 (2006)

    Google Scholar 

  45. Yang, T., Kecman, V.: Adaptive local hyperplane classification. Neurocomputing 71(13-15), 3001–3004 (2008)

    Article  Google Scholar 

  46. Segata, N., Blanzieri, E.: Fast and scalable local kernel machines. Journal Machine Learning Research 11, 1883–1926 (2010)

    MathSciNet  MATH  Google Scholar 

  47. Cheng, H., Tan, P.N., Jin, R.: Efficient algorithm for localized support vector machine. IEEE Transactions on Knowledge and Data Engineering 22(4), 537–549 (2010)

    Article  Google Scholar 

  48. Kecman, V., Brooks, J.: Locally linear support vector machines and other local models. In: The 2010 International Joint Conference on Neural Networks (IJCNN), pp. 1–6 (2010)

    Google Scholar 

  49. Ladicky, L., Torr, P.H.S.: Locally linear support vector machines. In: Getoor, L., Scheffer, T. (eds.) Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28-July 2, pp. 985–992. Omnipress (2011)

    Google Scholar 

  50. Gu, Q., Han, J.: Clustered support vector machines. In: Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2013, Scottsdale, AZ, USA, April 29-May 1. JMLR Proceedings, vol. 31, pp. 307–315 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Information Technology, Can Tho University, No 1, Ly Tu Trong Street, Ninh Kieu District, Can Tho City, Vietnam

    Thanh-Nghi Do

Authors
  1. Thanh-Nghi Do
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thanh-Nghi Do .

Editor information

Editors and Affiliations

  1. LITA - UFR MIM, University of Lorraine – Metz, Metz, France

    Hoai An Le Thi

  2. Institute of Informatics & Wrocław University of Technology, Wrocław, Poland

    Ngoc Thanh Nguyen

  3. Department of Networked Systems and Services, Budapest University of Technology and Economics, Budapest, Hungary

    Tien Van Do

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Do, TN. (2015). Non-linear Classification of Massive Datasets with a Parallel Algorithm of Local Support Vector Machines. In: Le Thi, H., Nguyen, N., Do, T. (eds) Advanced Computational Methods for Knowledge Engineering. Advances in Intelligent Systems and Computing, vol 358. Springer, Cham. https://doi.org/10.1007/978-3-319-17996-4_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-17996-4_21

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-17995-7

  • Online ISBN: 978-3-319-17996-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation