Skip to main content

Advertisement

Springer Nature Link
Log in

Support vector analysis of large-scale data based on kernels with iteratively increasing order

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

This study presents an efficient approach for large-scale data training. To deal with the rapid growth of training complexity for big data analysis, a novel mechanism, which utilizes fast kernel ridge regression (Fast KRR) and ridge support vector machines (Ridge SVMs), is proposed in this study. Firstly, Fast KRR based on low-order intrinsic-space computation is developed. Preliminary support vectors are located by using Fast KRR. Subsequently, the system iteratively removes indiscriminant data until a Ridge SVM with a high-order kernel can accommodate the data size and generate a hyperplane. To speed up the removal of indiscriminant data, quick intrinsic-matrix rebuilding is devised in the iteration. Experiments on three databases were carried out for evaluating the proposed method. Moreover, different percentages of data removal were examined in the test. The results show that the performance is enhanced by as high as 78–152 folds. Besides, the mechanisms still maintain the accuracy. These findings thereby demonstrate the effectiveness of the proposed idea.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Kung S-Y (2014) Kernel methods and machine learning. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  2. Chen B-W, Chen C-Y, Wang J-F (2013) Smart homecare surveillance system: behavior identification based on state transition support vector machines and sound directivity pattern analysis. IEEE Trans Syst Man Cybern Syst 43(6):1279–1289

    Article  Google Scholar 

  3. Guo Q, Chen B-W, Jiang F, Ji X, Kung S-Y (2015) Efficient divide-and-conquer classification based on feature-space decomposition. Retrieved from http://arxiv.org/abs/1501.07584

  4. Kung S-Y, Wu P-Y (2012) On efficient learning and classification kernel methods. In: Proceedings of 2012 IEEE international conference on acoustics, speech, and signal processing (ICASSP 2012), Kyoto, pp 2065–2068

  5. Divide and conquer algorithms. Wikipedia, the free encyclopedia. Retrieved from http://en.wikipedia.org/wiki/Divide_and_conquer_algorithms

  6. Chang EY, Zhu K, Wang H, Bai H, Li J, Qiu Z, Cui H (2007) PSVM: parallelizing support vector machines on distributed computers. In: Proceedings of 21st annual conference neural information processing system (NIPS 2007). Vancouver, pp 257–264

  7. Gu Q, Han J (2013) Clustered support vector machines. In: Proceedings of 16th international conference artificial intelligence and statistics (AISTATS). Scottsdale, pp 307–315

  8. Zhang Y, Duchi J, Wainwright M (2013) Divide and conquer kernel ridge regression. In: Proceedings of conference on learning theory (Colt 2013), Princeton, pp 592–617

  9. Hsieh C-J, Si S, Dhillon IS (2013) A divide-and-conquer solver for kernel support vector machines. In: Proceedings of 31st international conference on machine learning (ICML 2013). Beijing

  10. Lin C-J, Moré JJ (1999) Incomplete Cholesky factorizations with limited memory. SIAM J Sci Comput 21(1):24–45

    Article  MathSciNet  MATH  Google Scholar 

  11. Ralaivola L, d’Alché-Buc F (2001) Incremental support vector machine learning: a local approach. In: Proceedings of international conference on artificial neural networks (ICANN 2001). Vienna, pp 322–330

  12. Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceedings of 7th ACM international conference on knowledge discovery and data mining (SIGKDD 2001), San Francisco, pp 77–86

  13. Cauwenberghs G, Poggio T (2000) Incremental and decremental support vector machine learning: In: Proceedings of 14th annual conference neural information processing system (NIPS 2000) Denver, pp 409–415

  14. Diehl CP, Cauwenberghs G (2003) SVM incremental learning, adaptation and optimization. In: Proceedings of international joint conference on neural networks (IJCNN 2003), Portland, pp 2685–2690

  15. Laskov P, Gehl C, Krüger S, Müller K-R (2006) Incremental support vector learning: analysis, implementation and applications. J Mach Learn Res 7:1909–1936

    MathSciNet  MATH  Google Scholar 

  16. Karasuyama M, Takeuchi I (2010) Multiple incremental decremental learning of support vector machines. IEEE Trans Neural Netw 21(7):1048–1059

    Article  Google Scholar 

  17. Shilton A, Palaniswami M, Ralph D, Tsoi AC (2005) Incremental training of support vector machines. IEEE Trans Neural Netw 16(1):114–131

    Article  Google Scholar 

  18. Platt JC (1999) Fast training of support vector machines using sequential minimal optimization. In: Schölkopf B, Burges CJC, Smola AJ (eds) Advances in kernel methods: support vector learning. MIT Press, Cambridge

  19. Yu H-F, Hsieh C-J, Chang K-W, Lin C-J (2010) Large linear classification when data cannot fit in memory. In: Proceedings of 16th ACM SIGKDD international conference on knowledge discovery and data mining (KDD 2010). Washington, pp 833–842

  20. Hsieh C-J, Chang K-W, Lin C-J, Keerthi SS, Sundararajan S (2008) A dual coordinate descent method for large-scale linear SVM. In: Proceedings of 25th international conference on machine learning. Helsinki, pp 408–415

Download references

Acknowledgments

The authors would like to thank Dr. Pei-Yuan Wu for providing source codes. This work was supported in part by the Ministry of Science and Technology, the Republic of China under Grant No. 103-2917-I-564-058. Part of this research is sponsored by the Beijing Key Laboratory of Mobile Computing and Pervasive  Device, Institute of Computing Technology, Chinese Academy of Sciences, under the open project No. 2015-4.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Princeton University, Princeton, USA

    Bo-Wei Chen, Xinyu He & Sun-Yuan Kung

  2. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

    Wen Ji

  3. Department of Multimedia, Sungkyul University, Anyang, South Korea

    Seungmin Rho

Authors
  1. Bo-Wei Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xinyu He
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Wen Ji
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Seungmin Rho
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Sun-Yuan Kung
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Bo-Wei Chen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, BW., He, X., Ji, W. et al. Support vector analysis of large-scale data based on kernels with iteratively increasing order. J Supercomput 72, 3297–3311 (2016). https://doi.org/10.1007/s11227-015-1404-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-015-1404-1

Keywords