A Novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression
Introduction
Support vector machines (SVMs) have been successfully applied to a number of applications such as including handwriting recognition, particle identification (e.g., muons), digital images identification (e.g., face identification), text categorization, bioinformatics (e.g., gene expression), function approximation and regression, and database marketing, and so on. Although SVMs have become more widely employed to forecast time-series data (Tay and Cao, 2001, Cao, 2003, Kim, 2003) and to reconstruct dynamically chaotic systems (Müller et al., 1997, Mukherjee et al., 1997, Mattera and Haykin, 1999, Kulkarni et al., 2003), a highly effective model can only be built after the parameters of SVMs are carefully determined (Duan, Keerthi, & Poo, 2003).
Min and Lee (2005) stated that the optimal parameter search on SVM plays a crucial role in building a prediction model with high prediction accuracy and stability. The kernel-parameters are the few tunable parameters in SVMs controlling the complexity of the resulting hypothesis (Cristianini, Campell, & Taylor, 1999). Shawkat and Kate (2007) pointed out that selecting the optimal degree of a polynomial kernel is critical to ensure good generalization of the resulting support vector machine model. They proposed an automatic selection for determining the optimal degree of polynomial kernel in SVM by Bayesian and Laplace approximation method estimation and a rule based meta-learning approach. In addition, to construct an efficient SVM model with RBF kernel, two extra parameters: (a) sigma squared and (b) gamma, have to be carefully predetermined. However, few studies have been devoted to optimizing the parameter values of SVMs. Evolutionary algorithms often have to solve optimization problems in the presence of a wide range of problems (Dastidar et al., 2005, Shin et al., 2005, Yaochu and Branke, 2005, Zhang et al., 2005). In these algorithms, genetic algorithms (GAs) have been widely and successfully applied to various types of optimization problems in recent years (Goldberg, 1989, Fogel, 1994, Cao, 2003, Alba and Dorronsoro, 2005, Alba and Dorronsoro, 2005; Aurnhammer and Tonnies, 2005, Venkatraman and Yen, 2005, Hokey et al., 2006, Cao and Wu, 1999, McCall, 2005). Therefore, this paper proposes a hybrid genetic-based SVR model, HGA-SVR, which can automatically optimize the SVR parameters integrating the real-valued genetic algorithm (RGA) and integer genetic algorithm, for increasing the predictive accuracy and capability of generalization compared with traditional machine learning models.
In addition, a wide range of approaches including time-varying splines (Harvey & Koopman, 1993), multiple regression models (Ramanathan, Engle, Granger, Vahid-Araghi, & Brace, 1997), judgmental forecasts, artificial neural networks (Hippert & Pedreira, 2001) and SVMs (Chen et al., 2004, Tian and Noore, 2004) have been employed to forecast electricity load. One of the most crucial demands for the operation activities of power systems is short-term hourly load forecasting and the extension to several days in the future. Improving the accuracy of short-term load forecasting (STLF) is becoming even more significant than before due to the changing structure of the power utility industry (Tian & Noore, 2004). SVMs have been applied to STLF and performed well. Unfortunately, there is still no consensus as to the perfect approach to electricity demand forecasting (Taylor & Buizza, 2003).
Several studies have proposed optimization methods which used a genetic algorithm for optimizing the SVR parameter values. To overcome the problem of SVR parameters, a GA-SVR has been proposed in a earlier paper (Hsu, Wu, Chen, & Peng, 2006) to take advantage of the GAs optimization technique. However, few studies have focused on concurrently optimizing the type of SVR kernel function and the parameters of SVR kernel function. The present study proposed a novel and specialized hybrid genetic algorithm for optimizing all the SVR parameters simultaneously. Our proposed method was applied to predicting maximum electrical daily load and its performance was analyzed. An actual case of forecasting maximum electrical daily load is illustrated to show the improvement in predictive accuracy and capability of generalization achieved by our proposed HGA-SVR model.
The remainder of this paper is organized as follows. The research gap for obtaining optimal parameters in SVR is reviewed and discussed in Section 2. Section 3 details the proposed HGA-SVR, ideas and procedures. In Section 4 an experimental example for predicting the electricity load is described to demonstrate the proposed method. Discussions are presented in Section 5 and conclusions are drawn in the final Section.
Section snippets
Basic ideas of methods for obtaining optimal parameters in SVR
SVR is a promising technique for data classification and regression (Vapnik, 1998). We briefly introduce the basic idea of SVR in the Section 2.1. To design an effective model, the values of the essential parameters in SVR must be chosen carefully in advance (Duan et al., 2003). Thus, various approaches to determine these values are discussed in Section 2.2. Although many optimization methods have been proposed, GAs is well suited to the concurrent manipulation of models with varying
Design of the hybrid genetic-based SVR (HGA-SVR) model for improving predictive accuracy
In this section, we describe the design of our proposed novel HGA-SVR model. The optimization process of HGA-SVR is introduced in the first section. The basic idea of non-linear SVR model is described in the next section. The design of chromosome representations, fitness function and genetic operators in our novel HGA-SVR are discussed in the final sections.
Experimental example for predicting electricity load
In this section, the effectiveness of the proposed HGA-SVR model was demonstrated by forecasting the daily electricity loading problem as announced on the ’Worldwide Competition within the EUNITE Network1’. The set problem was to predict the maximum daily electricity load for January 1999 using
Conclusions
This study proposed a novel hybrid genetic algorithm for dynamically optimizing all the essential parameters of SVR. Our experimental results demonstrated the successful application of our proposed new model, HGA-SVR, for the complex forecasting problem. It demonstrated that it increased the electricity load forecasting accuracy more than any other model employed in the EUNITE network competition. Specifically, the new HGA-SVR model can successfully identify all the optimal values of the SVR
Acknowledgement
This work was supported by National Science Council of the Republic of China under Grant No. NSC 95-2416-H-147-005.
References (45)
Kernel methods: A survey of current techniques
Neurocomputing
(2002)Support vector machines experts for time series forecasting
Neurocomputing
(2003)- et al.
A comparison of tests of nonlinear cointegration with application to the predictability of US interest rates using the term structure
International Journal of Forecasting
(2004) - et al.
Evaluation of simple performance measures for tuning SVM hyperparameters
Neurocomputing
(2003) Financial time series forecasting using support vector machines
Neurocomputing
(2003)- et al.
Control of chaotic dynamical systems using support vector machines
Physics Letters A
(2003) - et al.
Fast and accurate power dispatch using a relaxed genetic algorithm and a local gradient technique
Expert Systems with Applications
(2000) Genetic Algorithms for Modelling and Optimization
Journal of Computational & Applied Mathematics
(2005)- et al.
Bankruptcy prediction using support vector machine with optimal choice of kernel function parameters
Expert Systems with Applications
(2005) - et al.
Short-run forecast of electricity loads and peaks
International Journal of Forecasting
(1997)
Application of support vector machines in financial time series forecasting
OMEGA The International Journal of Management Science
Using weather ensemble predictions in electricity demand forecasting
International Journal of Forecasting
The exploration/exploitation tradeoff in dynamic cellular genetic algorithms
IEEE Transactions on Evolutionary Computation
The exploration/exploitation tradeoff in dynamic cellular genetic algorithms
IEEE Transactions on Evolutionary Computation
A genetic algorithm for automated horizon correlation across faults in seismic images
IEEE Transactions on Evolutionary Computation
Optimization of control parameters in genetic algorithms: A stochastic approach
International Journal of Systems Science
Load forecasting using support vector machines: A study on EUNITE competition 2001
I EEE Transactions on Power Systems
An introduction to support vector machines
Dynamically adapting kernels in support vector machines
Advances in Neural Information Processing Systems
Speciation as automatic categorical modularization
IEEE Transactions on Evolutionary Computation
A synthesis system for analog circuits based on evolutionary search and topological reuse
IEEE Transactions on Evolutionary Computation
Cited by (255)
A conceptual metaheuristic-based framework for improving runoff time series simulation in glacierized catchments
2024, Engineering Applications of Artificial IntelligenceInterpretable machine learning for predicting the strength of 3D printed fiber-reinforced concrete (3DP-FRC)
2023, Journal of Building EngineeringPredicting the direction of financial dollarization movement with genetic algorithm and machine learning algorithms: The case of Turkey
2023, Expert Systems with ApplicationsMeta-Heuristic Search Optimization and its application to Time Series Forecasting Model
2022, Intelligent Systems with Applications