Skip to main content
SpringerLink
Log in

A non-biased form of least squares support vector classifier and its fast online learning

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

As an effective learning technique based on structural risk minimization, SVM has been confirmed an useful tool in many machine learning fields. With the increase in application requirement for some real-time cases, such as fast prediction and pattern recognition, the online learning based on SVM gradually becomes a focus. But the common SVM has disadvantages in classifier’s bias and the computational complexity of online modeling, resulting in the reduction in classifier’s generality and the low learning speed. Therefore, an non-biased least square support vector classifier(LSSVC) model is proposed in this paper by improving the form of structure risk. Also, a fast online learning algorithm using Cholesky factorization is designed based on this model according to the characteristic of the non-biased kernel extended matrix in the model’s dynamic change process. In this way, the calculation of Lagrange multipliers is simplified, and the time of online learning is greatly reduced. Simulation results testify that the non-biased LSSVC has good universal applicability and better generalization capability, at the same time, the algorithm has a great improvement on learning speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Suykens JAK, Vandewalle J (1999) Least square support vector machines classifiers. Neural Process Lett 9(3):293–300

    Article  MathSciNet  Google Scholar 

  2. Ralaivola L (2001) Incremental support vector machine learning: a local approach. In: Proceedings of the international on conference on artificial neural networks, Vienna, Austria. pp 322–329

  3. Ruping S (2002) Incremental learning with support vector machines. Dortmund University, Dortmund: Technical Report TR 18

  4. Shilton A, Palaniswami M, Ralph D, Tsoi AC (2005) Incremental training of support vector machines. IEEE Trans Neural Netw 16(1):114–131

    Article  Google Scholar 

  5. Lau KW, Wu QH (2003) Online training of support vector classifier. Pattern Regres 36(8):1913–1920

    Article  MATH  Google Scholar 

  6. Ma J, Theiler J, Perkens S (2003) Accurate on-line support vector regression. Neural Comput Appl 15:2683–2703

    Article  MATH  Google Scholar 

  7. Kivinen J, Smola AJ, Williamson RC (2004) Online learning with kernels. IEEE Trans Signal Process 52:2165–2176

    Article  MathSciNet  Google Scholar 

  8. Vishwanathan SVN, Schraudolph NN, Smola AJ (2006) Step size adaptation in reproducing kernel Hilbert space. J Mach Learn Res 7:1107–1133

    MathSciNet  Google Scholar 

  9. Martin M (2002) Online support vector machines for function approximation. Politecnica University, Catalunya, Spain: Technical Report LSI-02-11-R

  10. Zhang HR, Wang XD (2006) Incremental and online learning algorithm for regression least square vector machine. Chin J Comput 29(3):400–406

    Google Scholar 

  11. Fan YG, Li P, Song ZH (2006) Dynamic weighted least square support vector machines. Control Decis 21(10):1129–1133

    MathSciNet  MATH  Google Scholar 

  12. Suykens JAK, Vandewalle J (2000) Recurrent least squares support vector machines. IEEE Trans Circuits Syst 47(7):1109–1114

    Article  Google Scholar 

  13. Cesa-Bianchi N, Conconi A, Gentile C (2004) On the generalization ability of on-line learning algorithms. IEEE Trans Inf Theory 50(9):2050–2057

    Article  MathSciNet  Google Scholar 

  14. Crammer K, Dekel O, Keshet J, Shalev-Shwartz S, Singer Y (2006) Online passive-aggressive algorithms. J Mach Learn Res 7:551–585

    MathSciNet  Google Scholar 

  15. Fung G, Mangasarian O (2001) Proximal support vector machine classifiers In: Proceedings KDD-2001, knowledge discovery and data mining, San Francisco, CA

  16. Fung G, Mangasarian O (2001) Incremental support vector machine classification, Data Mining Institute Technical Report 01-08. Computer Sciences Department, University of Wisconsin

  17. Liu QG, He Q, Shi ZZ (2007) Incremental nonlinear proximal support vector machine. In: Proceedings of ISNN’07 4493: 336C341

  18. Gene HG, Charles FV (1996) Matrix computations. The Johns Hopkins University Press, Maryland

    MATH  Google Scholar 

  19. Seeger M (2005) Low Rank Updates for the Cholesky Decomposition. Technical Report, Max Planck Society, Tuebingen, Germany

  20. Ojeda F, Suykens JAK, Moor BD (2008) Low rank updated LS-SVM classifiers for fast variable selection. Neural Netw 21:437–449

    Article  Google Scholar 

  21. Vijayakumar S (1999) Sequential support vector classifiers and regression. In: Proceedings of international conference on soft computing, Genoa, Italy. pp 610–619

  22. Zhang HR, Zhang CJ, Wang XD (2006) A new support vector machine and its learning algorithm. In: Proceedings of the 6th world congress on control and automation, Dalian, China. pp 2820–2824

  23. Engel Y, Mannor S, Meir R (2002) Sparse online greedy support vector regression. In: Proceedings of European conference on machine learning. Spring, Berlin, pp 84–96

  24. Csato L, Opper M (2002) Sparse on-line Gaussian processes. Neural Comput Appl 14(3):641–668

    Article  MATH  Google Scholar 

  25. http://ida.first.fraunhofer.de/projects/benchprojects/bench

  26. Navia-Vzquez A, Prez-Cruz F, Arts-Rodriguez A, Figueiras-Vidal A (2004) Unbiased support vector classifiers. In: Proceedings of the IEEE signal processing society workshop. pp 183–192

  27. Navia-Vzquez A, Prez-Cruz F, Arts-Rodriguez A, Figueiras-Vidal A (2004) Advantages of unbiased support vector classifiers for data mining applications. J VLSI Signal Process 37:223–235

    Article  Google Scholar 

  28. Zheng DL (2006) Research on kernel methods in machine learning. Ph.D. thesis, Tsinghua University

  29. Wang HQ, Sun FC, Zhao ZT, Cai YN (2007) SAR image ATR using SVM with a low dimensional combined feature. In: Proceedings of SPIE MIPPR’07 6786:67862J

  30. Debnath R, Takahide N, Takahashi H (2004) A decision based one-against-one method for multi-class support vector machine. Pattern Anal Appl 7(2):164–175

    MathSciNet  Google Scholar 

Download references

Acknowledgments

This work was jointly supported by the National Science Fund for Distinguished Young Scholars (Grant No: 60625304), the National Natural Science Foundation of China (Grants No: 60621062, 60504003), the National Key Project for Basic Research of China (Grant No: 2007CB311003) and the Specialized Research Fund for the Doctoral Program of Higher Education (Grant No: 20050003049).

Author information

Authors and Affiliations

  1. Xi’an Research Institute of Hi-Tech, 710025, Xi’an, Shannxi, China

    Hong-Qiao Wang & Yan-Ning Cai

  2. Department of Computer Science and Technology, State Key Lab of Intelligent Technology and Systems, Tsinghua University, 100084, Beijing, China

    Hong-Qiao Wang & Fu-Chun Sun

Authors
  1. Hong-Qiao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan-Ning Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fu-Chun Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hong-Qiao Wang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, HQ., Cai, YN. & Sun, FC. A non-biased form of least squares support vector classifier and its fast online learning. Neural Comput & Applic 20, 1075–1085 (2011). https://doi.org/10.1007/s00521-010-0517-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-010-0517-y

Keywords

Search

Navigation