Skip to main content
SpringerLink
Log in

A novel multi-strategy DE algorithm for parameter optimization in support vector machine

  • Published:
Journal of Intelligent Information Systems Aims and scope Submit manuscript

Abstract

Support vector machine (SVM) is a powerful technique in pattern classification, but its performance is highly dependent on its parameters. In this paper, a new SVM optimized by a novel differential evolution (DE) with a hybrid parameter setting strategy and a population size adaptation method is proposed and simplified as FDE-PS-SVM. In the hybrid parameter setting strategy, the SVM parameter offspring are generated by DE operators with evolutionary parameters that are fixed or with the ones generated by fuzzy logic inference (FLI) according to a given probability. In the population size adaptation method, the population size is shrunk gradually during the search, which tries to balance the diversity and concentration ability of the algorithm to find better SVM parameters. Some benchmark data sets are used to evaluate the proposed algorithm. Experimental results show that the two proposed strategies are effective to search for better SVM parameters while the proposed FDE-PS-SVM algorithm outperforms other algorithms published in other literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Adankon, M.M., & Cheriet, M. (2009). Model selection for the LS-SVM. Application to handwriting recognition. Pattern Recognition, 42(12), 3264–3270.

    Article  MATH  Google Scholar 

  • Ahmad, A.S., Hassan, M.Y., Abdullah, M.P., Rahman, H.A., Hussin, F., Abdullah, H., Saidur, R. (2014). A review on applications of ANN and SVM for building electrical energy consumption forecasting. Renewable and Sustainable Energy Reviews, 33, 102–109.

    Article  Google Scholar 

  • Alon, U., Barkai, N., Notterman, D.A., Gish, K., Ybarra, S., Mack, D., et al. (1999). Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the National Academy of Sciences of the United States of America, 96(12), 6745–6750.

    Article  Google Scholar 

  • Aydin, I., Karakose, M., Akin, E. (2011). A multi-objective artificial immune algorithm for parameter optimization in support vector machine. Applied Soft Computing, 11(1), 120–129.

    Article  Google Scholar 

  • Brest, J., Greiner, S., Boskovic, B., Mernik, M., Zumer, V. (2006). Self-adapting control parameters in differential evolution: A comparative study on numerical benchmark problems. IEEE Transactions on Evolutionary Computation, 10(6), 646–657.

    Article  Google Scholar 

  • Burges, C.J.C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2), 121–167.

    Article  Google Scholar 

  • Chang, C.C., & Lin, C.J. (2011). Libsvm: a library for support vector machines. Acm Transactions on Intelligent Systems & Technology, 2(3), 27:1 – 27:27.

    Article  Google Scholar 

  • Chang, C.C., Hsu, C.W., Lin, C.J. (2016). A practical guide to support vector classification. https://www.csie.ntu.edu.tw/cjlin/papers/guide/guide.pdf.

  • Chapelle, O., Vapnik, V., Bousquet, O., Mukherjee, S. (2002). Choosing multiple parameters for support vector machines. Machine Learning, 46(1-3), 131–159.

    Article  MATH  Google Scholar 

  • Das, S., & Suganthan, P.N. (2011). Differential evolution: a survey of the state-of-the-art. IEEE Transactions on Evolutionary Computation, 15(1), 4–31.

    Article  Google Scholar 

  • Dua, D., & Graff, C. (2019). UCI machine learning repository. http://archive.ics.uci.edu/ml, University of California, Irvine, School of Information and Computer Sciences.

  • Duan, K., Keerthi, S.S., Poo, A.N. (2003). Evaluation of simple performance measures for tuning svm hyperparameters. Pattern Recognition, 51, 41–59.

    Google Scholar 

  • Eitrich, T., & Lang, B. (2006). Efficient optimization of support vector machine learning parameters for unbalanced datasets. Journal of Computational & Applied Mathematics, 196(2), 425–436.

    Article  MathSciNet  MATH  Google Scholar 

  • Guyon, I., Weston, J., Barnhill, S., Vapnik, V. (2002). Gene selection for cancer classification using support vector machines. Machine Learning, 46(1-3), 389–422.

    Article  MATH  Google Scholar 

  • Hsu, C.W., & Lin, C.J. (2002). A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks, 13(4), 41–425.

    Google Scholar 

  • Huang, C.L. (2009). Aco-based hybrid classification system with feature subset selection and model parameters optimization. Neurocomputing, 73(1-3), 438–448.

    Article  Google Scholar 

  • Huang, C.L., & Wang, C.J. (2006). A ga-based feature selection and parameters optimization for support vector machines. Expert Systems with Applications, 31(2), 231–240.

    Article  Google Scholar 

  • Keerthi, S.S., & Lin, C.J. (2003). Asymptotic behaviors of support vector machines with gaussian kernel. Neural Computation, 15(7), 1667–1689.

    Article  MATH  Google Scholar 

  • Keerthi, S.S., Sindhwani, V., Chapelle, O. (2007). An efficient method for gradient-based adaptation of hyperparameters in svm models. Advances in Neural Information Processing Systems 19:Proceedings of the 2006 Conference, 42(12), 673–680.

    Google Scholar 

  • Khan, N.M., Ksantini, R., Ahmad, I.S., Boufama, B. (2012). A novel svm+nda model for classification with an application to face recognition. Pattern Recognition, 45 (1), 66–79.

    Article  MATH  Google Scholar 

  • Li, S.T., Kwok, J.T., Zhu, H.L. (2003). Texture classification using the support vector machines. Pattern Recognition, 36(12), 2883–2893.

    Article  MATH  Google Scholar 

  • Li, J., Ding, L.X., Xing, Y. (2013). Differential evolution based parameters selection for support vector machine. In: International Conference on Computational Intelligence & Security, pp. 284–288.

  • Lin, H.T., & Lin. C.J. (2003). A study on sigmoid kernels for svm and the training of non-psd kernels by smo-type methods. http://www.csie.ntu.edu.tw/cjlin/papers/tanh.pdf.

  • Lin, S.W., Ying, K.C., Chen, S.C., Lee, Z.J. (2008). Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Systems with Applications, 35(4), 1817–1824.

    Article  Google Scholar 

  • Liu, J.H., & Lampinen, J. (2003). Population size adaptation for differential evolution algorithm using fuzzy logic. Intelligent Systems Design and Applications, 42 (12), 3264–3270.

    Google Scholar 

  • Liu, J., & Lampinen, J. (2005). A fuzzy adaptive differential evolution algorithm. Soft Computing, 9(6), 448–462.

    Article  MATH  Google Scholar 

  • Liu, R.J., Wang, Y.H., Baba, T., Masumoto, D., Nagata, S. (2008). Svm-based active feedback in image retrieval using clustering and unlabeled data. Pattern Recognition, 41(8), 2645–2655.

    Article  MATH  Google Scholar 

  • Olatomiwa, L., Mekhilef, S., Shamshirband, S., Mohammadi, K., Petković, D., Sudheer, C. (2015). A support vector machine–firefly algorithm-based model for global solar radiation prediction. Solar Energy, 115, 632–644.

    Article  Google Scholar 

  • Qin, A.K., & Suganthan, P.N. (2005). Self-adaptive differential evolution algorithm for numerical optimization. 2005 IEEE Congress on Evolutionary Computation, 2, 1785–1791.

    Article  Google Scholar 

  • Qin, A.K., Huang, V.L., Suganthan, P.N. (2009). Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Transactions on Evolutionary Computation, 13(2), 398–417.

    Article  Google Scholar 

  • Santos, G.S.D., Luvizotto, L.G.J., Mariani, V.C., Coelho, L.D.S. (2012). Least squares support vector machines with tuning based on chaotic differential evolution approach applied to the identification of a thermal process. Expert Systems with Applications, 39(5), 4805–4812.

    Article  Google Scholar 

  • Sarker, R.A., Elsayed, S.M., Ray, T. (2014). Differential evolution with dynamic parameters selection for optimization problems. IEEE Transactions on Evolutionary Computation.

  • Scholkopf, B., Guyon, I., Weston, J. (2001). Statistical learning and kernel methods in bioinformatics. International Centre for Mechanical Sciences, 6(97), 111–120.

    Google Scholar 

  • Storn, R., & Price, K. (1997). Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 11(4), 341–359.

    Article  MathSciNet  MATH  Google Scholar 

  • Vapnik, V.N. (1995). The Nature of Statistical Learning Theory. New York: Springer.

    Book  MATH  Google Scholar 

  • Wolpert, D.H., & Macready, W.G. (1997). No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1), 67–82.

    Article  Google Scholar 

  • Wu, C.H., Tzeng, G.H., Goo, Y.J., Fang, W.C. (2007). A real-valued genetic algorithm to optimize the parameters of support vector machine for predicting bankruptcy.

  • Xia, S.X., Lin, R., Cui, X., Shan, J. (2016). The application of orthogonal test method in the parameters optimization of fPEMFCg under steady working condition. International Journal of Hydrogen Energy, 41(26), 11380–11390.

    Article  Google Scholar 

  • Yuana, S., & Chua, F. (2007). Fault diagnosis based on support vector machines with parameter optimisation by artificial immunisation algorithm. Mechanical Systems and Signal Processing, 21(3), 1318–1330.

    Article  Google Scholar 

  • Zhang, J., & Sanderson, A.C. (2009). Jade: Adaptive differential evolution with optional external archive. IEEE Transactions on Evolutionary Computation, 13(5), 945–958.

    Article  Google Scholar 

  • Zhang, X.Y., Zhou, J.Z., Wang, C.Q., Li, C.S., Song, L.X. (2012). Multi-class support vector machine optimized by inter-cluster distance and self-adaptive deferential evolution. Applied Mathematics and Computation, 218(9), 4973–4987.

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang, X.Y., Qiu, D.Y., Chen, F.A. (2015). Support vector machine with parameter optimization by a novel hybrid method and its application to fault diagnosis. Neurocomputing, 149, 641–651.

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by the Fundamental Research Funds for the Central Universities (x2zd-D2192280), the National Natural Science Foundation of China (Grant No.61573146), the National Science and the Applied Science and Technology Research and Development Special Fund Project of Guangdong Province, China (Grant No.2015B010133003), the Natural Science Foundation of Guangdong Province, China (Grant No.2016A030313454).

Author information

Authors and Affiliations

  1. School of Automation Science & Engineering, South China University of Technology, Guangzhou, China

    Kangjing Li, Jiaxiang Luo, Yueming Hu & Shipeng Li

  2. Engineering Research Center of Ministry of Education for Precision Electronic Manufacturing Equipment, Guangzhou, China

    Kangjing Li, Jiaxiang Luo, Yueming Hu & Shipeng Li

  3. Guangdong Provincial Engineering Laboratory for Advanced Chip Intelligent Packaging Equipment, Guangzhou, China

    Kangjing Li, Jiaxiang Luo, Yueming Hu & Shipeng Li

Authors
  1. Kangjing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiaxiang Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yueming Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shipeng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiaxiang Luo.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, K., Luo, J., Hu, Y. et al. A novel multi-strategy DE algorithm for parameter optimization in support vector machine. J Intell Inf Syst 54, 527–543 (2020). https://doi.org/10.1007/s10844-019-00573-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10844-019-00573-w

Keywords

Search

Navigation