Abstract
In this paper, we propose a novel multi-parametric kernel Support Vector Regression algorithm (SVMr) optimized with an evolutionary technique, specially well suited for forecasting problems. The multi-parametric SVMr model and the evolutionary algorithm proposed are both described in detail in the paper. In addition, several new bounds for the multi-parametric kernel considered are obtained, in such a way that the SVMr hyper-parameters’ search space is reduced. We present experimental evidences of the good performance of the evolutionary algorithm for optimizing the multi-parametric kernel, when compared to a standard SVMr with a Grid Search approach. Specifically, results in different real regression problems from public repositories are obtained, and also a real application focused on the short-term temperature prediction at Barcelona’s airport. The results obtained have shown the good performance of the multi-parametric kernel approach both in accuracy and computation time.
Similar content being viewed by others
References
Abe S (2005) Training of support vector machines with Mahalanobis kernels. Lect Notes Comput Sci 3697:571–576
Akay MF (2009) Support vector machines combined with feature selection for breast cancer diagnosis. Expert Syst Appl 36(2):3240–3247
Asuncion A, Newman DJ (2007) UCI Machine Learning Repository 2007.University of California, School of Information and Computer Science, Irvine, CA. http://www.ics.uci.edu/~mlearn/MLRepository.html
Brodic D (2010) Optimization of the anisotropic Gaussian kernel for text segmentation and parameter extraction. Theor Comput Sci IFIP Adv Inf Commun Technol 323:140–152
Chang CC, Lin C-J (2011) LIBSVM : a library for support vector machines. ACM Trans Intell Syst Technol 2:1–27
Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46:131–159
Eiben AE, Smith JE (2003) Introduction to evolutionary computing. Springer, Berlin
Fauvel M, Villa A, Chanussot J, Benediktsson JA (2010) Mahalanobis kernel for the classification of hyperspectral images. In: Proceedings of the IEEE international geoscience and remote sensing symposium, Honolulu, Hawaii, pp 3724–3727
Friedrichs F, Igel C (2005) Evolutionary tuning of multiple SVM parameters. Neurocomputing 64:107–117
Gascón-Moreno J, Ortiz-García EG, Salcedo-Sanz S, Paniagua-Tineo A, Saavedra-Moreno B, Portilla-Figueras JA (2011) Multi-parametric Gaussian kernel function optimization for \(\epsilon\)-SVMr using a genetic algorithm, IWANN 2011, Lecture Notes in Computer Science, vol 6692, pp 113–120
Gijsberts A, Metta G, Rothkrantz L (2010) Evolutionary optimization of least-squares support vector machines. Ann Inf Syst 8(Special Issue on Data Mining):277–297
He W, Wang Z, Jiang H (2008) Model optimizing and feature selecting for support vector regression in time series forecasting. Neurocomputing 72(1–3):600–611
Hou S, Li Y (2009) Short-term fault prediction based on support vector machines with parameter optimization by evolution strategy. Expert Syst Appl 36(10):12383–12391
Ortiz-Garcia EG, Salcedo-Sanz S, Pérez-Bellido AM, Portilla-Figueras JA (2009) Improving the training time of support vector regression algorithms through novel hyper-parameters search space reductions. Neurocomputing 72(1–3):3683–3691
Pai PF, Hong WC, Lee YS (2005) Determining parameters of support vector machines by genetic algorithms—applications to reliability prediction. Int J Oper Res 2(1):1–7
Phienthrakul T, Kijsirikul B (2005) Evolutionary strategies for multi-scale radial basis function kernels in support vector machines. In: Proceedings of the 2005 conference on genetic and evolutionary computation, GECCO ’05
Phienthrakul T, Kijsirikul B (2010) Evolutionary strategies for hyperparameters of support vector machines based on multi-scale radial basis function kernels. Soft Comput 14(7):681–699
Rieger C, Zwicknagl B (2009) Deterministic error analysis of support vector regression and related regularized kernel methods. J Mach Learn Res 10:2115–2132
Salcedo-Sanz S, Ortiz-García G Emilio, Pérez-Bellido AM, Portilla-Figueras A, Prieto L (2011) Short term wind speed prediction based on evolutionary support vector regression algorithms. Expert Syst Appl 38(4):4052–4057
Shamsheyeva A, Sowmya A (2004) The anisotropic Gaussian kernel for SVM classification of HRCT images of the lung. In: Proceedings of the intelligent sensors, sensor networks and information processing conference
Shao X, Cherkassky V (1999) Multi-resolution support vector machine. In: Proceedings of international joint conference on neural networks 2, pp 1065–1070
Shitong W, Jiagang Z, Chung FL, Qing L, Dewen H (2005) Theoretically optimal parameter choices for support vector regression machines with noisy input. Soft Comput 9(10):732–741
Smola AJ, Schölkopf B (1998) A tutorial on support vector regression. Stat Comput
Smola AJ, Murata N, Scholkopf B, Muller K (1998) Asymptotically optimal choice of \(\epsilon\)-loss for support vector machines. In: Proceedings of the 8th international conference on artificial neural networks, perspectives in neural computing
StatLib DataSets Archive (2012). http://lib.stat.cmu.edu/datasets
Villa A, Fauvel M, Chanussot J, Gamba P, Benediktsson JA (2008) Gradient Optimization for multiple kernel’s parameters in support vector machines classification. In: IEEE international geoscience and remote sensing symposium, pp 224–227
Wang S, Zhu J, Chung FL, Dewen Hu (2006) Experimental study on parameter choices in norm-r support vector regression machines with noisy input. Soft Comput 10(3):219–223
Wu CL, Chau KW, Li YS (2008) River stage prediction based on a distributed support vector regression. J Hydrol 358(1–2):96–111
Wu GH, Tzeng GH, Lin RH (2009) A novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression. Expert Syst Appl 36(3):4725–4735
Yang Z, Guo J, Xu W, Nie X, Wang J, Lei J (2006) Multi-scale support vector machine for regression estimation. In: Lecture Notes in Computer Science, vol 3971, pp 1030–1037
Zhao YP, Sun JG (2011) Multikernel semiparametric linear programming support vector regression. Expert Syst Appl 38(3):1611–1618
Zheng D, Wang J, Zhao Y (2006) Non-flat function estimation with a multi-scale support vector regression. Neurocomputing 70:420–429
Zhou L, Lai KK, Yu L (2009) Credit scoring using support vector machines with direct search for parameters selection. Soft Comput 13(2):149–155
Acknowledgments
This work has been partially supported by Spanish Ministry of Science and Innovation, under project number ECO2010-22065-C03-02.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gascón-Moreno, J., Ortiz-García, E.G., Salcedo-Sanz, S. et al. Evolutionary optimization of multi-parametric kernel \(\epsilon\)-SVMr for forecasting problems. Soft Comput 17, 213–221 (2013). https://doi.org/10.1007/s00500-012-0886-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-012-0886-5