Skip to main content

Advertisement

SpringerLink
Log in

A novel multi-classifier based on a density-dependent quantized binary tree LSSVM and the logistic global whale optimization algorithm

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The least squares support vector machine (LSSVM) is a useful binary classifier, but its performance is limited due to the lack of sparseness. The density-dependent quantized LSSVM (DSM) with quantized input data can increase the sparseness to effectively accomplish binary classification. However, the DSM cannot be directly used in multi-classification applications for most practical data-classification problems. We propose a novel multi-classifier based on a density-dependent quantized binary tree LSSVM (DBSM) and the logistic global whale optimization algorithm (LWA) to improve multi-classification accuracy and computational efficiency. The DBSM consists of multiple DSM classifiers, which hierarchically divide data according to a modified binary tree architecture. The tree architecture is constructed quickly and correctly with the quantized data instead of the original input data. An appropriate initial population of DBSM parameters is generated by using a logistic map and an improved opposition-based learning strategy. Then, the DBSM parameters are optimized by the whale optimization algorithm integrated with the gbest-guided artificial bee colony algorithm. According to the experimental results, the DBSM solves multi-classification problems faster than the one-versus-one based support vector machine (OVO-SVM) and the one-versus-all based LSSVM without sacrificing accuracy. The LWA precisely finds the optimal DBSM parameters without a heavy computational burden, in contrast to recent optimization algorithms. The proposed classifier achieves a 3.39% higher accuracy and consumes 52.83% less time than the genetic algorithm-based OVO-SVM. These results prove that the LWA-DBSM can complete multi-class classification tasks precisely and quickly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Wu XD, Zhu XQ, Wu GQ, Ding W (2014) Data mining with big data. IEEE Trans Knowl Data Eng 26(1):97–107. https://doi.org/10.1109/tkde.2013.109

    Article  Google Scholar 

  2. Wang JX, Tan DP, Cao B, Fan J, Deep S (2020) Independent path-based process recommendation algorithm for improving biomedical process modeling. Electronics Letter 2:1–13. https://doi.org/10.1049/el.2019.3978

    Article  Google Scholar 

  3. Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Letters 9(3):293–300. https://doi.org/10.1023/a:1018628609742

    Article  Google Scholar 

  4. Tang JJ, Liu F, Zou YJ, Zhang WB, Wang YH (2017) An improved fuzzy neural network for traffic speed prediction considering periodic characteristic. IEEE Trans Intell Transpo Sys 18(9):2340–2350. https://doi.org/10.1109/tits.2016.2643005

    Article  Google Scholar 

  5. Ticay-Rivas JR, del Pozo-Banos M, Eberhard WG, Alonso JB, Travieso CM (2013) Spider specie identification and verification based on pattern recognition of it cobweb. Expert Syst Appl 40 (10):4213–4225. https://doi.org/10.1016/j.eswa.2013.01.024

    Article  Google Scholar 

  6. Bui DT, Tuan TA, Hoang ND, Thanh NQ, Nguyen DB, Liem NV, Pradhan B (2017) Spatial prediction of rainfall-induced landslides for the Lao Cai area (Vietnam) using a hybrid intelligent approach of least squares support vector machines inference model and artificial bee colony optimization. Landslides 14 (2):447–458. https://doi.org/10.1007/s10346-016-0711-9

    Article  Google Scholar 

  7. Deng W, Yao R, Zhao HM, Yang XH, Li GY (2019) A novel intelligent diagnosis method using optimal LS-SVM with improved PSO algorithm. Soft Comput 23(7):2445–2462. https://doi.org/10.1007/s00500-017-2940-9

    Article  Google Scholar 

  8. Li CB, Li SK, Liu YQ (2016) A least squares support vector machine model optimized by moth-flame optimization algorithm for annual power load forecasting. Appl Intell 45(4):1166–1178. https://doi.org/10.1007/s10489-016-0810-2

    Article  Google Scholar 

  9. Oliveira SAF, Gomes JPP, Neto ARR (2018) Sparse least-squares support vector machines via accelerated segmented test: A dual approach. Neurocomputing 321:308–320. https://doi.org/10.1016/j.neucom.2018.08.081

    Article  Google Scholar 

  10. Mall R, Suykens JAK (2015) Very sparse LSSVM reductions for large-scale data. IEEE Trans Neural Netw Learn Syst 26(5):1086–1097. https://doi.org/10.1109/tnnls.2014.2333879

    Article  MathSciNet  Google Scholar 

  11. Karsmakers P, Pelckmans K, De Brabanter K, Van Hamme H, Suykens JAK (2011) Sparse conjugate directions pursuit with application to fixed-size kernel models. Mach Learn 85(1-2):109–148. https://doi.org/10.1007/s10994-011-5253-8

    Article  MathSciNet  MATH  Google Scholar 

  12. Zhou SS (2016) Sparse LSSVM in Primal Using Cholesky Factorization for Large-Scale Problems. IEEE Trans Neural Netw Learn Syst 27(4):783–795. https://doi.org/10.1109/tnnls.2015.2424684

    Article  MathSciNet  Google Scholar 

  13. Silva DA, Silva JP, Neto ARR (2015) Novel approaches using evolutionary computation for sparse least square support vector machines. Neurocomputing 168:908–916. https://doi.org/10.1016/j.neucom.2015.05.034

    Article  Google Scholar 

  14. Nan SY, Sun L, Chen BD, Lin ZP, Toh KA (2017) Density-dependent quantized least squares support vector machine for large data sets. IEEE Trans Neural Netw Learn Syst 28(1):94–106. https://doi.org/10.1109/tnnls.2015.2504382

    Article  Google Scholar 

  15. Deng F, Guo S, Zhou R, Chen J (2017) Sensor multifault diagnosis with improved support vector machines. IEEE Trans Autom Sci Eng 14(2):1053–1063. https://doi.org/10.1109/tase.2015.2487523

    Article  Google Scholar 

  16. Hamedi M, Salleh SH, Noor AM (2015) Facial neuromuscular signal classification by means of least square support vector machine for MuCI. Appl Soft Comput 30:83–93. https://doi.org/10.1016/j.asoc.2015.01.034

    Article  Google Scholar 

  17. Zhang XK, Ding SF, Sun TF (2016) Multi-class LSTMSVM based on optimal directed acyclic graph and shuffled frog leaping algorithm. Int J Mach Learn Cybern 7 (2):241–251. https://doi.org/10.1007/s13042-015-0435-5

    Article  Google Scholar 

  18. Khemchandani R, Sharma S (2016) Robust least squares twin support vector machine for human activity recognition. Appl Soft Comput 47:33–46. https://doi.org/10.1016/j.asoc.2016.05.025

    Article  Google Scholar 

  19. Tomar D, Agarwal S (2015) A comparison on multi-class classification methods based on least squares twin support vector machine. Knowl.-Based Syst 81:131–147. https://doi.org/10.1016/j.knosys.2015.02.009

    Article  Google Scholar 

  20. Cevikalp H, Elmas M, Ozkan S (2018) Large-scale image retrieval using transductive support vector machines. Comput Vis Image Underst 173:2–12. https://doi.org/10.1016/j.cviu.2017.07.004

    Article  Google Scholar 

  21. Lee Y, Lee J (2015) Binary tree optimization using genetic algorithm for multiclass support vector machine. Expert Syst Appl 42(8):3843–3851. https://doi.org/10.1016/j.eswa.2015.01.022

    Article  Google Scholar 

  22. Bogawar PS, Bhoyar KK (2018) An improved multiclass support vector machine classifier using reduced hyper-plane with skewed binary tree. Appl Intell 48(11):4382–4391. https://doi.org/10.1007/s10489-018-1218-y

    Article  Google Scholar 

  23. Yu LA, Dai W, Tang L, Wu JQ (2016) A hybrid grid-GA-based LSSVR learning paradigm for crude oil price forecasting. Neural Computing & Applications 27(8):2193–2215. https://doi.org/10.1007/s00521-015-1999-4

    Article  Google Scholar 

  24. Ghamisi P, Benediktsson JA (2015) Feature Selection Based on Hybridization of Genetic Algorithm and Particle Swarm Optimization. IEEE Geoscience And Remote Sensing Letters 12(2):309–313. https://doi.org/10.1109/lgrs.2014.2337320

    Article  Google Scholar 

  25. Pahasa J, Ngamroo I (2011) Least square support vector machine for power system stabilizer design using wide area phasor measurements. Int J Innov Comp Inf Control 7(7B):4487–4501

    Google Scholar 

  26. Yuan XH, Chen C, Yuan YB, Huang YH, Tan QX (2015) Short-term wind power prediction based on LSSVM-GSA model. Energy Conversion and Management 101:393–401. https://doi.org/10.1016/j.enconman.2015.05.065

    Article  Google Scholar 

  27. Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Softw 95:51–67. https://doi.org/10.1016/j.advengsoft.2016.01.008

    Article  Google Scholar 

  28. Sun WZ, Wang JS (2017) Elman neural network soft-sensor model of conversion velocity in polymerization process optimized by Chaos Whale optimization algorithm. IEEE Access 5:13062–13076. https://doi.org/10.1109/access.2017.2723610

    Article  Google Scholar 

  29. Yan ZH, Sha JX, Liu B, Tian W, Lu JP (2018) An ameliorative whale optimization algorithm for multi-objective optimal allocation of water resources in Handan, China. Water 10(1):29. 10.3390/w10010087

    Article  Google Scholar 

  30. Abd Elaziz M, Oliva D (2018) Parameter estimation of solar cells diode models by an improved opposition-based whale optimization algorithm. Energy Conversion and Management 171:1843–1859. https://doi.org/10.1016/j.enconman.2018.05.062

    Article  Google Scholar 

  31. Abdel-Basset M, Manogaran G, El-Shahat D, Mirjalili S (2018) A hybrid whale optimization algorithm based on local search strategy for the permutation flow shop scheduling problem. Futur Gener Comp Syst 85:129–145. https://doi.org/10.1016/j.future.2018.03.020

    Article  Google Scholar 

  32. Krishna JV, Naidu GA, Upadhayaya N (2018) A Lion-Whale optimization-based migration of virtual machines for data centers in cloud computing. Int J Commun Syst 31(8):18. https://doi.org/10.1002/dac.3539

    Article  Google Scholar 

  33. Zhu GP, Kwong S (2010) Gbest-guided artificial bee colony algorithm for numerical function optimization. Appl Math Comput 217(7):3166–3173. https://doi.org/10.1016/j.amc.2010.08.049

    Article  MathSciNet  MATH  Google Scholar 

  34. Williams CKI, Seeger M (2001) Using the Nystrom method to speed up kernel machines. In: Leen T K, Dietterich T G, Tresp V (eds) Advances In Neural Information Processing Systems, vol 13. MIT Press, Cambridge, pp 682–688

  35. Xue PN, Gao XJ, Wang P, Qi YS (2015) The multi-classification algorithm combining an improved binary tree with SVM and its application of fault diagnosis. 2015 IEEE International Conference on Information And Automation. IEEE, New York

  36. Wang XD, Wu CM (2005) Using improved SVM decision tree to classify HRRP. In: Proceedings of 2005 international conference on machine learning and cybernetics, vol 1–9. IEEE, New York

  37. Zheng TY, Luo WL (2019) An enhanced lightning attachment procedure optimization with quasi-opposition-based learning and dimensional search strategies. Comput Intell Neurosci 2019:24. https://doi.org/10.1155/2019/1589303

    Article  Google Scholar 

  38. Rahnamayan S, Tizhoosh HR, Salama MMA (2007) A novel population initialization method for accelerating evolutionary algorithms. Comput Math Appl 53(10):1605–1614. https://doi.org/10.1016/j.camwa.2006.07.013

    Article  MathSciNet  MATH  Google Scholar 

  39. Cui LZ, Zhang K, Li GH, Fu XH, Wen ZK, Lu N, Lu J (2018) Modified Gbest-guided artificial bee colony algorithm with new probability model. Soft Comput 22(7):2217–2243. https://doi.org/10.1007/s00500-017-2485-y

    Article  Google Scholar 

  40. Jain S, Shukla S, Wadhvani R (2018) Dynamic selection of normalization techniques using data complexity measures. Expert Syst Appl 106:252–262. https://doi.org/10.1016/j.eswa.2018.04.008

    Article  Google Scholar 

  41. Ling H, Qian CX, Mang WC, Liang CY, Chen HC (2019) Combination of support vector machine and K-Fold cross validation to predict compressive strength of concrete in marine environment. Construction And Building Materials 206:355–363. https://doi.org/10.1016/j.conbuildmat.2019.02.071

    Article  Google Scholar 

  42. Khan MMR, Arif RB, Siddique MA, Oishe MR (2018) Study and observation of the variation of accuracies of KNN, SVM, LMNN, ENN algorithms on eleven different datasets from UCI machine learning repository. In: 2018 4th International conference on electrical engineering and information & communication technology. international conference on electrical engineering and information communication technology. IEEE, New York, pp 124–129

  43. Liu Y, Bi JW, Fan ZP (2017) A method for multi-class sentiment classification based on an improved one-vs-one (OVO) strategy and the support vector machine (SVM) algorithm. Inform Sci 394:38–52. https://doi.org/10.1016/j.ins.2017.02.016

    Article  Google Scholar 

  44. Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61. https://doi.org/10.1016/j.advengsoft.2013.12.007

    Article  Google Scholar 

  45. Zhang X, Xu YT, Yu CY, Heidari AA, Li SM, Chen HL, Li CY (2020) Gaussian mutational chaotic fruit fly-built optimization and feature selection. Expert Syst Appl 141:14. https://doi.org/10.1016/j.eswa.2019.112976

    Article  Google Scholar 

  46. Zheng HB, Liao RJ, Grzybowski S, Yang LJ (2011) Fault diagnosis of power transformers using multi-class least square support vector machines classifiers with particle swarm optimisation. IET Electr Power Appl 5(9):691–696. https://doi.org/10.1049/iet-epa.2010.0298

    Article  Google Scholar 

  47. Gao X, Hou J (2016) An improved SVM integrated GS-PCA fault diagnosis approach of Tennessee Eastman process. Neurocomputing 174:906–911. https://doi.org/10.1016/j.neucom.2015.10.018

    Article  Google Scholar 

  48. Huang MM, Lin RS, Huang S, Xing TF (2017) A novel approach for precipitation forecast via improved K-nearest neighbor algorithm. Adv Eng Inform 33:89–95. https://doi.org/10.1016/j.aei.2017.05.003

    Article  Google Scholar 

  49. Nepomuceno EG, Martins SAM, Silva BC, Amaral GFV, Perc M (2018) Detecting unreliable computer simulations of recursive functions with interval extensions. Appl Math Comput 329:408–419. https://doi.org/10.1016/j.amc.2018.02.020

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work is supported in part by the NSFC-Zhejiang Joint Fund for the Integration of Industrialization and Informatization (No. U1509212) and in part by the Research on Public Welfare Technology Application Projects of Zhejiang Province (No. LGG18E050023).

Author information

Authors and Affiliations

  1. Key Laboratory of E&M, Zhejiang University of Technology, Ministry of Education, Hangzhou, Zhejiang Province, 310012, People’s Republic of China

    Jiaoliao Chen, Xingai Zhuo, Fang Xu, Jiacai Wang & Libin Zhang

  2. Department of Mechanical Engineering, Lassonde School of Engineering, York University, Toronto, M2J4A6, Canada

    Dan Zhang

Authors
  1. Jiaoliao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xingai Zhuo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiacai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Libin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fang Xu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, J., Zhuo, X., Xu, F. et al. A novel multi-classifier based on a density-dependent quantized binary tree LSSVM and the logistic global whale optimization algorithm. Appl Intell 50, 3808–3821 (2020). https://doi.org/10.1007/s10489-020-01736-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01736-x

Keywords

Search

Navigation