Abstract
Support vector machine (SVM) is a powerful technique in pattern classification, but its performance is highly dependent on its parameters. In this paper, a new SVM optimized by a novel differential evolution (DE) with a hybrid parameter setting strategy and a population size adaptation method is proposed and simplified as FDE-PS-SVM. In the hybrid parameter setting strategy, the SVM parameter offspring are generated by DE operators with evolutionary parameters that are fixed or with the ones generated by fuzzy logic inference (FLI) according to a given probability. In the population size adaptation method, the population size is shrunk gradually during the search, which tries to balance the diversity and concentration ability of the algorithm to find better SVM parameters. Some benchmark data sets are used to evaluate the proposed algorithm. Experimental results show that the two proposed strategies are effective to search for better SVM parameters while the proposed FDE-PS-SVM algorithm outperforms other algorithms published in other literature.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Adankon, M.M., & Cheriet, M. (2009). Model selection for the LS-SVM. Application to handwriting recognition. Pattern Recognition, 42(12), 3264–3270.
Ahmad, A.S., Hassan, M.Y., Abdullah, M.P., Rahman, H.A., Hussin, F., Abdullah, H., Saidur, R. (2014). A review on applications of ANN and SVM for building electrical energy consumption forecasting. Renewable and Sustainable Energy Reviews, 33, 102–109.
Alon, U., Barkai, N., Notterman, D.A., Gish, K., Ybarra, S., Mack, D., et al. (1999). Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the National Academy of Sciences of the United States of America, 96(12), 6745–6750.
Aydin, I., Karakose, M., Akin, E. (2011). A multi-objective artificial immune algorithm for parameter optimization in support vector machine. Applied Soft Computing, 11(1), 120–129.
Brest, J., Greiner, S., Boskovic, B., Mernik, M., Zumer, V. (2006). Self-adapting control parameters in differential evolution: A comparative study on numerical benchmark problems. IEEE Transactions on Evolutionary Computation, 10(6), 646–657.
Burges, C.J.C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2), 121–167.
Chang, C.C., & Lin, C.J. (2011). Libsvm: a library for support vector machines. Acm Transactions on Intelligent Systems & Technology, 2(3), 27:1 – 27:27.
Chang, C.C., Hsu, C.W., Lin, C.J. (2016). A practical guide to support vector classification. https://www.csie.ntu.edu.tw/cjlin/papers/guide/guide.pdf.
Chapelle, O., Vapnik, V., Bousquet, O., Mukherjee, S. (2002). Choosing multiple parameters for support vector machines. Machine Learning, 46(1-3), 131–159.
Das, S., & Suganthan, P.N. (2011). Differential evolution: a survey of the state-of-the-art. IEEE Transactions on Evolutionary Computation, 15(1), 4–31.
Dua, D., & Graff, C. (2019). UCI machine learning repository. http://archive.ics.uci.edu/ml, University of California, Irvine, School of Information and Computer Sciences.
Duan, K., Keerthi, S.S., Poo, A.N. (2003). Evaluation of simple performance measures for tuning svm hyperparameters. Pattern Recognition, 51, 41–59.
Eitrich, T., & Lang, B. (2006). Efficient optimization of support vector machine learning parameters for unbalanced datasets. Journal of Computational & Applied Mathematics, 196(2), 425–436.
Guyon, I., Weston, J., Barnhill, S., Vapnik, V. (2002). Gene selection for cancer classification using support vector machines. Machine Learning, 46(1-3), 389–422.
Hsu, C.W., & Lin, C.J. (2002). A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks, 13(4), 41–425.
Huang, C.L. (2009). Aco-based hybrid classification system with feature subset selection and model parameters optimization. Neurocomputing, 73(1-3), 438–448.
Huang, C.L., & Wang, C.J. (2006). A ga-based feature selection and parameters optimization for support vector machines. Expert Systems with Applications, 31(2), 231–240.
Keerthi, S.S., & Lin, C.J. (2003). Asymptotic behaviors of support vector machines with gaussian kernel. Neural Computation, 15(7), 1667–1689.
Keerthi, S.S., Sindhwani, V., Chapelle, O. (2007). An efficient method for gradient-based adaptation of hyperparameters in svm models. Advances in Neural Information Processing Systems 19:Proceedings of the 2006 Conference, 42(12), 673–680.
Khan, N.M., Ksantini, R., Ahmad, I.S., Boufama, B. (2012). A novel svm+nda model for classification with an application to face recognition. Pattern Recognition, 45 (1), 66–79.
Li, S.T., Kwok, J.T., Zhu, H.L. (2003). Texture classification using the support vector machines. Pattern Recognition, 36(12), 2883–2893.
Li, J., Ding, L.X., Xing, Y. (2013). Differential evolution based parameters selection for support vector machine. In: International Conference on Computational Intelligence & Security, pp. 284–288.
Lin, H.T., & Lin. C.J. (2003). A study on sigmoid kernels for svm and the training of non-psd kernels by smo-type methods. http://www.csie.ntu.edu.tw/cjlin/papers/tanh.pdf.
Lin, S.W., Ying, K.C., Chen, S.C., Lee, Z.J. (2008). Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Systems with Applications, 35(4), 1817–1824.
Liu, J.H., & Lampinen, J. (2003). Population size adaptation for differential evolution algorithm using fuzzy logic. Intelligent Systems Design and Applications, 42 (12), 3264–3270.
Liu, J., & Lampinen, J. (2005). A fuzzy adaptive differential evolution algorithm. Soft Computing, 9(6), 448–462.
Liu, R.J., Wang, Y.H., Baba, T., Masumoto, D., Nagata, S. (2008). Svm-based active feedback in image retrieval using clustering and unlabeled data. Pattern Recognition, 41(8), 2645–2655.
Olatomiwa, L., Mekhilef, S., Shamshirband, S., Mohammadi, K., Petković, D., Sudheer, C. (2015). A support vector machine–firefly algorithm-based model for global solar radiation prediction. Solar Energy, 115, 632–644.
Qin, A.K., & Suganthan, P.N. (2005). Self-adaptive differential evolution algorithm for numerical optimization. 2005 IEEE Congress on Evolutionary Computation, 2, 1785–1791.
Qin, A.K., Huang, V.L., Suganthan, P.N. (2009). Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Transactions on Evolutionary Computation, 13(2), 398–417.
Santos, G.S.D., Luvizotto, L.G.J., Mariani, V.C., Coelho, L.D.S. (2012). Least squares support vector machines with tuning based on chaotic differential evolution approach applied to the identification of a thermal process. Expert Systems with Applications, 39(5), 4805–4812.
Sarker, R.A., Elsayed, S.M., Ray, T. (2014). Differential evolution with dynamic parameters selection for optimization problems. IEEE Transactions on Evolutionary Computation.
Scholkopf, B., Guyon, I., Weston, J. (2001). Statistical learning and kernel methods in bioinformatics. International Centre for Mechanical Sciences, 6(97), 111–120.
Storn, R., & Price, K. (1997). Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 11(4), 341–359.
Vapnik, V.N. (1995). The Nature of Statistical Learning Theory. New York: Springer.
Wolpert, D.H., & Macready, W.G. (1997). No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1), 67–82.
Wu, C.H., Tzeng, G.H., Goo, Y.J., Fang, W.C. (2007). A real-valued genetic algorithm to optimize the parameters of support vector machine for predicting bankruptcy.
Xia, S.X., Lin, R., Cui, X., Shan, J. (2016). The application of orthogonal test method in the parameters optimization of fPEMFCg under steady working condition. International Journal of Hydrogen Energy, 41(26), 11380–11390.
Yuana, S., & Chua, F. (2007). Fault diagnosis based on support vector machines with parameter optimisation by artificial immunisation algorithm. Mechanical Systems and Signal Processing, 21(3), 1318–1330.
Zhang, J., & Sanderson, A.C. (2009). Jade: Adaptive differential evolution with optional external archive. IEEE Transactions on Evolutionary Computation, 13(5), 945–958.
Zhang, X.Y., Zhou, J.Z., Wang, C.Q., Li, C.S., Song, L.X. (2012). Multi-class support vector machine optimized by inter-cluster distance and self-adaptive deferential evolution. Applied Mathematics and Computation, 218(9), 4973–4987.
Zhang, X.Y., Qiu, D.Y., Chen, F.A. (2015). Support vector machine with parameter optimization by a novel hybrid method and its application to fault diagnosis. Neurocomputing, 149, 641–651.
Acknowledgements
This work was partially supported by the Fundamental Research Funds for the Central Universities (x2zd-D2192280), the National Natural Science Foundation of China (Grant No.61573146), the National Science and the Applied Science and Technology Research and Development Special Fund Project of Guangdong Province, China (Grant No.2015B010133003), the Natural Science Foundation of Guangdong Province, China (Grant No.2016A030313454).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, K., Luo, J., Hu, Y. et al. A novel multi-strategy DE algorithm for parameter optimization in support vector machine. J Intell Inf Syst 54, 527–543 (2020). https://doi.org/10.1007/s10844-019-00573-w
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10844-019-00573-w