Abstract
Support vector machine (SVM) is considered as one of the most powerful classifiers. They are parameterized models build upon the support vectors extracted during the training phase. One of the crucial tasks in the modeling of SVM is to select optimal values for its hyper-parameters, because the effectiveness and efficiency of SVM depend upon these parameters. This task of selecting optimal values for the SVM hyper-parameters is often called as the SVM model selection problem. Till now a lot of methods have been proposed to deal with this SVM model selection problem, but most of these methods consider the model selection problem in static environment only, where the knowledge about a problem does not change over time. In this paper we have proposed a framework to deal with SVM model selection problem in dynamic environment. In dynamic environment, knowledge about a problem changes over time due to which static optimum values for yper-parameters may degrade the performance of the classifier. For this there should be one efficient mechanism which can re-evaluate the optimal values of hyper-parameters when the knowledge about a problem changes. Our proposed framework uses multi-swarm-based optimization with exclusion and anti-convergence theory to select the optimal values for the SVM hyper-parameters in dynamic environment. The experiments performed using the proposed framework have shown better results in comparison with other techniques like traditional gird search, first grid search, PSO, chained PSO and dynamic model selection in terms of effectiveness and efficiency.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ayat N, Cheriet M, Suen C (2005) Automatic model selection for the optimization of SVM kernels. Pattern Recognit 38(10):1733–1745
Bansal S (2014) Optimal Golomb ruler sequence generation for FWM crosstalk elimination: soft computing versus conventional approaches. Appl Soft Comput 22:443–457
Bansal S, Gupta N, Singh AK (2017a) Nature–inspired metaheuristic algorithms to find near–OGR sequences for WDM channel allocation and their performance comparison. Open Math 15(1):520–547
Bansal S, Singh AK, Gupta N (2017b) Optimal Golomb ruler sequences generation for optical WDM systems: a novel parallel hybrid multi-objective bat algorithm. J Inst Eng (India) Ser B 98(1):43–64
Blackwell T (2005) Particle swarms and population diversity. Soft Comput 9(11):793–802
Blackwell TM, Bentley P (2002) Don’t push me! Collision avoiding swarms. In: Proceedings of congress on evolutionary computation, 2002, pp 1691–1696
Blackwell TM, Bentley P (2002) Dynamic search with charged swarms. In: Langdon WB et al (eds) Proceedings of genetic and computation conference, pp 19–26
Blackwell T, Branke J (2004) Multi-swarm optimization in dynamic environments. In: Raidl GR et al (eds) Applications of evolutionary computing. Series Lecture Notes in Computer Science, vol 3005. Springer, Berlin
Blackwell T, Branke J (2006a) Multiswarms, exclusion, and anti-convergence in dynamic environments. IEEE Trans Evol Comput 10(4):460–463
Blackwell T, Branke J (2006b) Multiswarms, exclusion, and anti-convergence in dynamic environments. IEEE Trans Evol Comput 10(4):462–471
Bottou L, Lin C-J (2007) Support vector machine solvers. Large Scale kernel Mach 3:301–320
Change CC, Lin CJ (2005) LIBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/cjlin/libsvm/
Chapelle O, Vapnik V (1999) Model selection for support vector machines. In: Advances in neural information processing systems, pp 230–236
Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1–3):131–159
Chatelain C, Adam S, Lecourtier Y, Heutte L, Paquet T (2007) Multiobjective optimization for SVM model selection. In: Proceedings of the 9th international conference on documentation analysis and recognition, 2007, pp 427–431
Chunhong Z, Licheng J (2004) Automatic parameters selection for SVM based on GA. In: Proceedings of the 5th world congress on intelligent control and automation, 2004, pp 1869–1872
Cohen G, Hilario M, Geissbuhler A (2004) Model selection for support vector classifiers via genetic algorithms. An application to medical decision support. In: Proceedings of the 5th international symposium on biological and medical data analysis, 2004, pp 200–211
Critianini N, Shawe- Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge
de Auza BF, de Carvalho ACPLF, Calvo R, Ishii RP (2006) Multiclass SVM model selection using particle swarm optimization. In: Proceedings of the 6th international conference on hybrid intelligence systems, 2006, pp 31–34
Hsu C-W, Lin C-J (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13:415–425
Hu X, Eberhart R (2002) Adaptive particle swarm optimization: detection and response to dynamic systems. In: Proceedings of Congress on Evolutionary Computation, 2002, pp 1666–1670
Huang C-M, Lee Y-J, Lin DK, Huang S-Y (2007) Model selection for support vector machines via uniform design. Comput Stat Data Anal 52(1):335–346
Janson S, Middendorf M (2004) A hierarchical particle swarm optimizer for dynamic optimization problems. In: Raidle GR (ed) Applications of evolutionary computing. Series Lecture Notes in Computer Science, vol 3005. Springer, Berlin, pp 513–524
Jiang M, Yuan X (2007) Construction and application of PSO–SVM model for personal credit scoring. In: Proceedings of the international conference on computational science. Lecture Notes in Computer Science, 2007, pp 158–161
Junfei L, Baolei Z (2014) Online learning algorithm of direct support vector machine for regression based on Cholesky factorization. In: 2014 international conference on information science, electronics and electrical engineering, vol 3. IEEE
Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE international conference neural network, pp 1942–1948
Kennedy J, Mendes R (2002) Population structure and particle swarm performance. In: Proceedings of congress on evolutionary computation, 2002, pp 1671–1676
Marcelo NK, Sabourin R, Maupin P (2012) A dynamic model selection strategy for support vector machine classifiers. Appl Soft Comput 12(2012):2550–2565
Parsopoulos KE, Vrahatis MN (2004) On the computational of all global minimizers through particle swarm optimization. IEEE Trans Evol Comput 8(3):211–224
Riedrichs F, Igel C (2004) Evolutionary tuning of multiple SVM parameters. In: Proceedings of the 12th European symposium on artificial neural networks, 2004, pp 519–524
Schoeman IL, Engelbrecht AP (2005) A parallel vector-based particle swarm optimizer. In: Ribeiro B, Albrecht RF, Dobnikar A, Pearson DW, Steele NC (eds) Adaptive and natural computing algorithms. Springer, Vienna, pp 268–271
Suttorp T, Igel C (2006) Multi-objective optimization of support vector machines. In: Jin Y (ed) Multi-objective machine learning. Studies in computational intelligence, vol 16. Springer, Berlin, Heidelberg, pp 199–220
Vapnik VN (1995) The nature of statistical learning theory, vol 5. Springer, New York
Vapnik VN (1998) Statistical learning theory. Wiley, New York, p 1998
Zhou X, Zhang X, Wang B (2016) Online support vector machine: a survey. In: Kim J, Geem Z (eds) Harmony search algorithm. Advances in intelligent systems and computing, vol 382. Springer, Berlin, pp 269–278
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Human participants and/or animals
The chapters do not contain any studies with human participants or animals performed by any of the authors
Informed consent
No human participants have been involved in the study
Additional information
Communicated by V. Loia.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kalita, D.J., Singh, S. SVM Hyper-parameters optimization using quantized multi-PSO in dynamic environment. Soft Comput 24, 1225–1241 (2020). https://doi.org/10.1007/s00500-019-03957-w
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-019-03957-w