Skip to main content
SpringerLink
Log in

An enhanced online LS-SVM approach for classification problems

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

In this paper, two novel approaches are proposed to improve the performance of online least squares support vector machine for classification problem. First, the parameters of support vector classifier model including kernel width parameter are simultaneously updated when a new sample arrives. In that model, kernel width parameter is a nonlinear term which cannot be estimated via least squares solution. Therefore, unscented Kalman filter is adopted to train all the parameters where Karush–Kuhn–Tucker conditions are satisfied. Second, a variable-size moving window, which is updated by an intelligent strategy, is proposed to construct the support vector set. Thus, the proposed model captures the dynamics of data quickly while precluding itself to become clumsy due to big amount of useless data. In addition, adaptive support vector set provides a lower computational load especially for the large data sets. Simultaneous training of the model parameters by unscented Kalman filter and intelligent update of support vector set provides a superior classification performance compared to the online support vector classification approaches in the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  • Amari S, Wu S (1999) Improving support vector machine classifiers by modifying kernel functions. Neural Netw 12(6):783–789

    Article  Google Scholar 

  • Bouhouche S, Yazid LL, Hocine S, Bast J (2010) Evaluation using online support-vector-machines and fuzzy reasoning. Application to condition monitoring of speeds rolling process. Control Eng Pract 18(9):1060–1068

    Article  Google Scholar 

  • Cauwenberghs G, Poggio T (2001) Incremental and decremental support vector machine learning. In: Advances in neural information processing systems, pp 409–415

  • Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1–3):131–159

    Article  MATH  Google Scholar 

  • Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  • Dahiya K, Chauhan VK, Sharma A (2017) Online support vector machine based on minimum euclidean distance. In: Proceedings of international conference on computer vision and image processing. Springer, pp 89–99

  • Davy M, Desobry F, Gretton A, Doncarli C (2006) An online support vector machine for abnormal events detection. Signal Process 86(8):2009–2025

    Article  MATH  Google Scholar 

  • Ertekin S, Leon Bottou C, Giles L (2011) Nonconvex online support vector machines. IEEE Trans Pattern Anal Mach Intell 33(2):368–381

    Article  Google Scholar 

  • Frie T-T, Cristianini N, Campbell C (1998) The Kernel-Adatron algorithm: a fast and simple learning procedure for support vector machines. In: Machine learning: proceedings of the fifteenth international conference (ICML’98), pp 188–196

  • Hao Z, Yu S, Yang X, Zhao F, Hu R, Liang Y (2004) Online LS-SVM learning for classification problems based on incremental chunk. In: International symposium on neural networks. Springer, pp 558–564

  • Hong W-C (2009) Chaotic particle swarm optimization algorithm in a support vector regression electric load forecasting model. Energy Convers Manag 50(1):105–117

    Article  Google Scholar 

  • Hong W-C, Dong Y, Wen YuZ, Li-Yueh C, Panigrahi BK (2013) Cyclic electric load forecasting by seasonal SVR with chaotic genetic algorithm. Int J Electr Power Energy Syst 44(1):604–614

    Article  Google Scholar 

  • Jian L, Shen S, Li J, Liang X, Li L (2016) Budget online learning algorithm for least squares SVM. IEEE Trans Neural Netw Learn Syst 99:1–12

    Article  Google Scholar 

  • Jiang Z, Liu C, Zhang G, Wang Y, Huang C, Liang J (2013) GPS/INS integrated navigation based on UKF and simulated annealing optimized SVM. In: Proceedings of the 78th IEEE vehicular technology conference, VTC Fall 2013, Las Vegas, NV, USA, September 2–5, pp 1–5

  • Joachims T (1999) Making large scale SVM learning practical. Technical report, Universität Dortmund

  • Jung T, Polani D (2006) Sequential learning with LS-SVM for large-scale data sets. In: International conference on artificial neural networks. Springer, pp 381–390

  • Kecman V, Melki G (2016) Fast online algorithms for support vector machines. SoutheastCon 2016:1–6

    Google Scholar 

  • Keerthi SS, Shevade SK, Bhattacharyya C, Murthy KRK (2001) Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Comput 13(3):637–649

    Article  MATH  Google Scholar 

  • Lau KW, Wu QH (2003) Online training of support vector classifier. Pattern Recogn 36(8):1913–1920

    Article  MATH  Google Scholar 

  • LeCun Y, Boser B, John SD, Donnie H, Richard EH, Wayne H, Lawrence DJ (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551

    Article  Google Scholar 

  • Lee MS, Moore AW (1994) Efficient algorithms for minimizing cross validation error. In: ICML, pp 190–198

  • Li Z, Outbib R, Giurgea S, Hissel D, Jemei S, Giraud A, Rosini S (2016) Online implementation of SVM based fault diagnosis strategy for PEMFC systems. Appl Energy 164:284–293

  • Li-Juan LI, Hong-Ye SU, Jian CHU (2007) Generalized predictive control with online least squares support vector machines. Acta Autom Sin 33(11):1182–1188

    MathSciNet  MATH  Google Scholar 

  • Lin CT, Yeh CM, Liang SF, Chung JF, Kumar N (2006) Support-vector-based fuzzy neural network for pattern classification. IEEE Trans Fuzzy Syst 14(1):31–41

    Article  Google Scholar 

  • Liu Y, Naiping H, Wang H, Li P (2009) Soft chemical analyzer development using adaptive least-squares support vector regression with selective pruning and variable moving window size. Ind Eng Chem Res 48(12):5731–5741

    Article  Google Scholar 

  • Ma J, Theiler J, Perkins S (2003) Accurate online support vector regression. Neural Comput 15:2683–2703

    Article  MATH  Google Scholar 

  • Manimala K, David IG, Selvi K (2015) A novel data selection technique using fuzzy C-means clustering to enhance SVM-based power quality classification. Soft Comput 19(11):3123–3144

    Article  Google Scholar 

  • Mu T, Nandi AK (2006) EKF based multiple parameter tuning system for a L2-SVM classifier. In: 2006 16th IEEE signal processing society workshop on machine learning for signal processing. IEEE, pp 229–233

  • Omitaomu OA, Jeong MK, Badiru AB, Hines JW (2007) Online support vector regression approach for the monitoring of motor shaft misalignment and feedwater flow rate. IEEE Trans Syst Man Cybern C 37(5):962–970

    Article  Google Scholar 

  • Osuna E, Freund R, Girosi F (1997) An improved training algorithm for support vector machines. In: Neural networks for signal processing VII. Proceedings of the 1997 IEEE signal processing society workshop, pp 276–285

  • Platt J (1998) Sequential minimal optimization: a fast algorithm for training support vector machines. Technical report

  • Qiong W, Liu W, Yang Y (2007) Time series online prediction algorithm based on least squares support vector machine. J Cent South Univ Technol 14(3):442–446

    Article  Google Scholar 

  • Ruping S (2001)Incremental learning with support vector machines. In: IEEE international conference on data mining, p 641

  • Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245

    Article  Google Scholar 

  • Schölkopf B, Simard P, Smola AJ, Vapnik V (1998) Prior knowledge in support vector kernels. In: Advances in neural information processing systems, pp 640–646

  • Suykens JAK, Joos V (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300

  • Suykens JAK, Van Gestel T, De Brabanter J (2002) Least squares support vector machines. World Scientific, Singapore

    Book  MATH  Google Scholar 

  • Tang H-S, Xue S-T, Chen R, Sato T (2006) Online weighted LS-SVM for hysteretic structural system identification. Eng Struct 28(12):1728–1735

    Article  Google Scholar 

  • Vapnik VN (1995) The nature of statistical learning theory. Springer, New York

    Book  MATH  Google Scholar 

  • Wan EA, Van Der Merwe R (2000) The unscented Kalman filter for nonlinear estimation. In: Proceedings of the IEEE 2000 adaptive systems for signal processing, communications, and control symposium, pp 153–158

  • Wang ZZZ, Mao J (2012) Adaptive tracking control based on online LS-SVM identifier. Int J Fuzzy Syst 14(2):330–336

    MathSciNet  Google Scholar 

  • Wang W, Zongben X, Weizhen L, Zhang X (2003) Determination of the spread parameter in the gaussian kernel for classification and regression. Neurocomputing 55(3–4):643–663

    Article  Google Scholar 

  • Yang X, Jie L, Zhang G (2010) Adaptive pruning algorithm for least squares support vector machine classifier. Soft Comput 14(7):667–680

    Article  MATH  Google Scholar 

  • Yu X, Wang X (2016) A novel hybrid classification framework using SVM and differential evolution. Soft Comput 21:4029–4044

    Article  Google Scholar 

Download references

Acknowledgements

This work is partly supported by Council of Pamukkale University Scientific Research Projects (BAP) Department.

Author information

Authors and Affiliations

  1. Faculty of Technology, Department of Mechatronics Engineering, Pamukkale University, Denizli, Turkey

    Erdem Dilmen

  2. Faculty of Engineering, Department of Electrical and Electronics Engineering, Pamukkale University, Denizli, Turkey

    Selami Beyhan

Authors
  1. Erdem Dilmen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Selami Beyhan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erdem Dilmen.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dilmen, E., Beyhan, S. An enhanced online LS-SVM approach for classification problems. Soft Comput 22, 4457–4475 (2018). https://doi.org/10.1007/s00500-017-2713-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-017-2713-5

Keywords

Search

Navigation