Abstract
In this paper, two novel approaches are proposed to improve the performance of online least squares support vector machine for classification problem. First, the parameters of support vector classifier model including kernel width parameter are simultaneously updated when a new sample arrives. In that model, kernel width parameter is a nonlinear term which cannot be estimated via least squares solution. Therefore, unscented Kalman filter is adopted to train all the parameters where Karush–Kuhn–Tucker conditions are satisfied. Second, a variable-size moving window, which is updated by an intelligent strategy, is proposed to construct the support vector set. Thus, the proposed model captures the dynamics of data quickly while precluding itself to become clumsy due to big amount of useless data. In addition, adaptive support vector set provides a lower computational load especially for the large data sets. Simultaneous training of the model parameters by unscented Kalman filter and intelligent update of support vector set provides a superior classification performance compared to the online support vector classification approaches in the literature.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Amari S, Wu S (1999) Improving support vector machine classifiers by modifying kernel functions. Neural Netw 12(6):783–789
Bouhouche S, Yazid LL, Hocine S, Bast J (2010) Evaluation using online support-vector-machines and fuzzy reasoning. Application to condition monitoring of speeds rolling process. Control Eng Pract 18(9):1060–1068
Cauwenberghs G, Poggio T (2001) Incremental and decremental support vector machine learning. In: Advances in neural information processing systems, pp 409–415
Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1–3):131–159
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297
Dahiya K, Chauhan VK, Sharma A (2017) Online support vector machine based on minimum euclidean distance. In: Proceedings of international conference on computer vision and image processing. Springer, pp 89–99
Davy M, Desobry F, Gretton A, Doncarli C (2006) An online support vector machine for abnormal events detection. Signal Process 86(8):2009–2025
Ertekin S, Leon Bottou C, Giles L (2011) Nonconvex online support vector machines. IEEE Trans Pattern Anal Mach Intell 33(2):368–381
Frie T-T, Cristianini N, Campbell C (1998) The Kernel-Adatron algorithm: a fast and simple learning procedure for support vector machines. In: Machine learning: proceedings of the fifteenth international conference (ICML’98), pp 188–196
Hao Z, Yu S, Yang X, Zhao F, Hu R, Liang Y (2004) Online LS-SVM learning for classification problems based on incremental chunk. In: International symposium on neural networks. Springer, pp 558–564
Hong W-C (2009) Chaotic particle swarm optimization algorithm in a support vector regression electric load forecasting model. Energy Convers Manag 50(1):105–117
Hong W-C, Dong Y, Wen YuZ, Li-Yueh C, Panigrahi BK (2013) Cyclic electric load forecasting by seasonal SVR with chaotic genetic algorithm. Int J Electr Power Energy Syst 44(1):604–614
Jian L, Shen S, Li J, Liang X, Li L (2016) Budget online learning algorithm for least squares SVM. IEEE Trans Neural Netw Learn Syst 99:1–12
Jiang Z, Liu C, Zhang G, Wang Y, Huang C, Liang J (2013) GPS/INS integrated navigation based on UKF and simulated annealing optimized SVM. In: Proceedings of the 78th IEEE vehicular technology conference, VTC Fall 2013, Las Vegas, NV, USA, September 2–5, pp 1–5
Joachims T (1999) Making large scale SVM learning practical. Technical report, Universität Dortmund
Jung T, Polani D (2006) Sequential learning with LS-SVM for large-scale data sets. In: International conference on artificial neural networks. Springer, pp 381–390
Kecman V, Melki G (2016) Fast online algorithms for support vector machines. SoutheastCon 2016:1–6
Keerthi SS, Shevade SK, Bhattacharyya C, Murthy KRK (2001) Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Comput 13(3):637–649
Lau KW, Wu QH (2003) Online training of support vector classifier. Pattern Recogn 36(8):1913–1920
LeCun Y, Boser B, John SD, Donnie H, Richard EH, Wayne H, Lawrence DJ (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551
Lee MS, Moore AW (1994) Efficient algorithms for minimizing cross validation error. In: ICML, pp 190–198
Li Z, Outbib R, Giurgea S, Hissel D, Jemei S, Giraud A, Rosini S (2016) Online implementation of SVM based fault diagnosis strategy for PEMFC systems. Appl Energy 164:284–293
Li-Juan LI, Hong-Ye SU, Jian CHU (2007) Generalized predictive control with online least squares support vector machines. Acta Autom Sin 33(11):1182–1188
Lin CT, Yeh CM, Liang SF, Chung JF, Kumar N (2006) Support-vector-based fuzzy neural network for pattern classification. IEEE Trans Fuzzy Syst 14(1):31–41
Liu Y, Naiping H, Wang H, Li P (2009) Soft chemical analyzer development using adaptive least-squares support vector regression with selective pruning and variable moving window size. Ind Eng Chem Res 48(12):5731–5741
Ma J, Theiler J, Perkins S (2003) Accurate online support vector regression. Neural Comput 15:2683–2703
Manimala K, David IG, Selvi K (2015) A novel data selection technique using fuzzy C-means clustering to enhance SVM-based power quality classification. Soft Comput 19(11):3123–3144
Mu T, Nandi AK (2006) EKF based multiple parameter tuning system for a L2-SVM classifier. In: 2006 16th IEEE signal processing society workshop on machine learning for signal processing. IEEE, pp 229–233
Omitaomu OA, Jeong MK, Badiru AB, Hines JW (2007) Online support vector regression approach for the monitoring of motor shaft misalignment and feedwater flow rate. IEEE Trans Syst Man Cybern C 37(5):962–970
Osuna E, Freund R, Girosi F (1997) An improved training algorithm for support vector machines. In: Neural networks for signal processing VII. Proceedings of the 1997 IEEE signal processing society workshop, pp 276–285
Platt J (1998) Sequential minimal optimization: a fast algorithm for training support vector machines. Technical report
Qiong W, Liu W, Yang Y (2007) Time series online prediction algorithm based on least squares support vector machine. J Cent South Univ Technol 14(3):442–446
Ruping S (2001)Incremental learning with support vector machines. In: IEEE international conference on data mining, p 641
Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245
Schölkopf B, Simard P, Smola AJ, Vapnik V (1998) Prior knowledge in support vector kernels. In: Advances in neural information processing systems, pp 640–646
Suykens JAK, Joos V (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300
Suykens JAK, Van Gestel T, De Brabanter J (2002) Least squares support vector machines. World Scientific, Singapore
Tang H-S, Xue S-T, Chen R, Sato T (2006) Online weighted LS-SVM for hysteretic structural system identification. Eng Struct 28(12):1728–1735
Vapnik VN (1995) The nature of statistical learning theory. Springer, New York
Wan EA, Van Der Merwe R (2000) The unscented Kalman filter for nonlinear estimation. In: Proceedings of the IEEE 2000 adaptive systems for signal processing, communications, and control symposium, pp 153–158
Wang ZZZ, Mao J (2012) Adaptive tracking control based on online LS-SVM identifier. Int J Fuzzy Syst 14(2):330–336
Wang W, Zongben X, Weizhen L, Zhang X (2003) Determination of the spread parameter in the gaussian kernel for classification and regression. Neurocomputing 55(3–4):643–663
Yang X, Jie L, Zhang G (2010) Adaptive pruning algorithm for least squares support vector machine classifier. Soft Comput 14(7):667–680
Yu X, Wang X (2016) A novel hybrid classification framework using SVM and differential evolution. Soft Comput 21:4029–4044
Acknowledgements
This work is partly supported by Council of Pamukkale University Scientific Research Projects (BAP) Department.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by V. Loia.
Rights and permissions
About this article
Cite this article
Dilmen, E., Beyhan, S. An enhanced online LS-SVM approach for classification problems. Soft Comput 22, 4457–4475 (2018). https://doi.org/10.1007/s00500-017-2713-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-017-2713-5