A novel hybrid algorithm for function approximation

doi:10.1016/j.eswa.2006.09.006

Expert Systems with Applications

Volume 34, Issue 1, January 2008, Pages 384-390

https://doi.org/10.1016/j.eswa.2006.09.006 Get rights and content

Abstract

This paper introduces a novel hybrid algorithm for function approximation. The proposed algorithm consists of a hybrid approach to develop Takagi and Sugeno’s fuzzy model for function approximation. In this paper, a coarse tuning based on Takagi and Sugeno’s fuzzy model is applied to identify the fuzzy structure, and also a fuzzy cluster validity index is utilized to determine the optimal number of clusters. To obtain a more precision model, genetic algorithm (GA) and particle swarm optimization (PSO) are performed to conduct fine-tuning for the obtained parameter set of the premise parts and consequent parts in the aforementioned fuzzy model. The proposed algorithm is successfully applied to three tested examples. Compared with other existing approaches in the literature, the proposed algorithm is very useful for modeling function approximation.

Introduction

Fuzzy model is one of the most important techniques in fuzzy logic which has advantages of excellent capability to deal with complex systems (Bezdek, 1981, Chuang et al., 2001, Yager and Filev, 1994). For building fuzzy model, the most important issue is the structure identification and parameters estimation. The structure identification concerns about the appropriate number of fuzzy rules, and the parameter estimation is related the calculation of parameters that represent a reliable system. Within various fuzzy model techniques, one of the most outstanding models was proposed by Takagi and Sugeno (1985). This fuzzy model is capable of describing a complex system using sufficient rules and training data (Chuang et al., 2001, Dave and Krishnapurum, 1997; Nozaki et al., 1997). However, this fuzzy model must define fuzzy subspaces in advance, and then the parameters in consequences are obtained through a Kalman filter type of estimation (Chuang et al., 2001). Various alternative approaches for modeling Takagi and Sugeno’s fuzzy rules have been proposed in the literature (Dickerson and Kosko, 1996, Frigui and Krishnapuram, 1999, Jang et al., 1997, Klawonn and Kruse, 1997, Kroll, 1996). In general, the input space is decomposed into fuzzy subspaces according to the input data and then the system or function approximation is approximated in each subspace by a simple linear regression function. The most drawbacks of the aforementioned approaches do not account for the interaction between input and output variables (Chuang et al., 2001). Recently, some authors have employed two phases of learning for fuzzy model: the coarse learning phase and the fine-tuning phase (Chuang et al., 2001; Kim et al., 1997–1998; Tsekouras et al., 2005). The idea is to formulate fuzzy structure with input and output variables of the system, and the parameter set of premise parts and consequent parts are approximately first determined. Moreover, the parameter set of premise parts and consequent are precisely adjusted by the fine-tuning phase to improve the modeling accuracy. However, the coarse tuning phase does not incorporate the validity index in fuzzy models, and the number of fuzzy rules is not determined by optimization algorithms. In the fine-tuning phase, the gradient descent approach is used to adjust the parameter set of premise parts and consequent parts. Due to the property of gradient descent approach, the found parameter set may be trapped in local optima (Jang et al., 1997, Lin and Lee, 1996, Michalewicz, 1994).

In this paper, a novel hybrid algorithm for function approximation is proposed. This paper is aimed toward to present a systematic approach for function approximation. The proposed algorithm is also composed of two phases: the coarse tuning phase and the fine tuning phase. The coarse tuning phase consists of structure identification and parameter estimation. Structure identification uses the well-known Xie-Beni (XB) fuzzy cluster validity index to determine the optimal number of fuzzy rules. The XB index is an optimizing index to study the relative merits of a portioned structure (Hanifi and Abdulkadir, 2006, Pakhira et al., 2005). Then the corresponding parameter set of premise parts and consequent parts are also roughly obtained in the coarse tuning phase. In the fine tuning phase, the parameter set of premise parts and consequent parts obtained from the coarse tuning phase are precisely adjusted to minimize the root mean square errors (RMSE) of the data set by a hybrid algorithm of genetic algorithm (GA) and particle swarm optimization (PSO).

The rest of this paper is outlined as follows. Section 2 describes the concept for Takagi and Sugeno’s fuzzy model, which will be used in this paper. Section 3 describes the basic GA and PSO concepts. Section 4, the structure of the proposed algorithm is discussed in detail. Section 5 reports the use of the proposed algorithm for modeling three examples, and the effectiveness of the proposed algorithm is illustrated. From simulation results, it is shown that the proposed algorithm outperforms other existing algorithms. Concluding remarks are presented in Section 6.

Section snippets

The concept of Takagi and Sugeno’s fuzzy model

The fuzzy model proposed by Takagi and Sugeno is one of the most efficient fuzzy models that can represent a general class of nonlinear systems or functions (Tsekouras et al., 2005). It is based on the feature that the input-output relations can be viewed as the expansion of piecewise linear partition. Typically, Takagi and Sugeno’s fuzzy model consists of IF–THEN rules that can be expressed as the following form: $\begin{matrix} R^{i} : If x_{1} is A_{1}^{i} (Ω_{1}^{i}) and x_{2} is A_{2}^{i} (Ω_{2}^{i}), \dots, x_{m} is A_{m}^{i} (Ω_{m}^{i}) \\ then y^{i} = f_{i} (x_{1}, x_{2}, \dots, x_{m}; {\overset{⇀}{a}}^{i}) = 1 + a_{1}^{i} x_{1} + \dots + \end{matrix}$

The introduction of genetic algorithm and particle swarm optimization

The proposed algorithm is based on genetic algorithm (GA) and particle swarm optimization (PSO) in fine tuning the values of parameter set obtained from the coarse tuning phase. The general idea is to combine the advantages of PSO and GA, the ability to exploit and explore the search space. In this section, the basic concepts of GA and PSO are introduced.

The proposed algorithm

The function approximation is to obtain a model $\hat{f}$ from a set of observations, ${(\vec{x} (1), y_{1}), (\vec{x} (2), y_{2}), \dots, (\vec{x} (N), y_{N})}$ with $\vec{x} (i) \in R^{m}$ and y_i ∈ R, where N is the number of training data, $\vec{x} (i) = [x_{1} (i), x_{2} (i), \dots, x_{m} (i)]$ is the ith input vector, and y_i is the desired output for the input $\vec{x} (i)$ . Basically, we want to construct the model $\hat{f}$ that can represent the real model in term of input–output relationships. In the proposed algorithm, the fuzzy model is generated by the coarse tuning phase, first. Then, a

Simulation results

In simulations, we need to identify a set of values. The size of the initial population for GA and PSO is 20. The crossover probability P_c = 0.5, mutation probability P_m = 0.02, MAXGEN = 5000, c₁ = 1, c₂ = 1, k₁ = 0.9, k₂ = 0.4 and l = 10. The values of min_r and max_r for GA are set as the maximum and the minimum values of input variables. Three examples are tested to verify the validity of the proposed algorithm. The first example is a nonlinear function (F1) taken from (Kim et al., 1997, Kim et al., 1997–1998

Conclusion

In the proposed algorithm, a novel hybrid algorithm for function approximation is proposed to simultaneously define fuzzy rules and find the parameters in the parameter set of premise and consequent parts. The number of fuzzy rules is determined to an optimum which is calculated using the XB validity index. In our approach, a fine-tuning algorithm based on GA and PSO is further employed to obtain a more precision model. The proposed algorithm is tested for various examples and indeed showed

References (28)

F. Klawonn et al.
Constructing a fuzzy controller from data
Fuzzy Sets and Systems
(1997)
A. Kroll
Identification of functional fuzzy models using multi-dimensional reference fuzzy sets
Fuzzy Sets and Systems
(1996)
Z.-J. Lee et al.
A genetic algorithm based robust learning credit assignment cerebellar model articulation controller
Applied Soft Computing
(2004)
K. Nozaki et al.
A simple but powerful method for generating fuzzy rules from numerical data
Fuzzy Sets and Systems
(1997)
M.K. Pakhira et al.
A study of some fuzzy cluster validity indices, genetic clustering and application to pixel classification
Fuzzy Sets and Systems
(2005)
G. Tsekouras et al.
A hierarchical fuzzy-clustering approach to fuzzy modeling
Fuzzy Sets and Systems
(2005)
J.C. Bezdek
Pattern recognition with fuzzy objective function algorithm
(1981)
C.-C. Chuang et al.
Robust TSK fuzzy modeling for function approximation with outliers
IEEE Transactions on Fuzzy Systems
(2001)
C.A.C. Coello et al.
Handling multiple objectives with particle swarm optimization
IEEE Transactions on Evolutionary Computation
(2004)
R.N. Dave et al.
Robust clustering methods: a unified view
IEEE Transactions on Fuzzy Systems
(1997)

J.A. Dickerson et al.

Fuzzy function approximation with ellipsoidal rules

IEEE Transactions on Systems Man and Cybernetics

(1996)

H. Frigui et al.

A robust competitive clustering algorithm with applications in computer vision

IEEE Transactions on Pattern Analysis and Machine Intelligence

(1999)

M. Gen et al.

Genitic algorithms and engineering design

(1997)

D.A. Goldberg

Genetic algorithms in search, optimization, and machine learning

(1989)

Cited by (36)

CFNN: Correlated fuzzy neural network
2015, Neurocomputing
Citation Excerpt :
In these methods, usually the number of fuzzy rules increases during training, so using a pruning method for removing redundant fuzzy rules, such as the method used in [22,55] is useful. In some methods, metaheuristic search methods based on swarm intelligence or evolutionary algorithms such as particle swarm optimization (PSO) and genetic algorithm (GA) are used to evolve the number of fuzzy rules and their parameters to optimize some objective functions [3,39,42,44,50,62]. A GA-based method proposed in [62] can evolve the number of fuzzy rules as well as all the membership function׳s (MF) parameters.
In this paper, a new fuzzy neural network model with correlated fuzzy rules (CFNN) based on the Levenberg–Marquardt (LM) optimization method is proposed. The proposed method is a new fuzzy network structure that is presented to approximate nonlinear functions especially the functions with high correlation between input variables with less number of fuzzy rules. A multivariable Gaussian fuzzy membership function is introduced that can consider the correlation between input variables and consequently it can model non-separable relations for interactive variables. The LM optimization method is used to learn parameters of both premise and consequent parts of the fuzzy rules. The suggested algorithm is successfully applied to seven tested examples including static function approximation, time-series prediction, non-linear dynamic system identification and a real-world complex regression problem. According to test observations; it can approximate nonlinear functions better than the other past algorithms with more compact structure and less number of fuzzy rules.
Cost estimation of plastic injection molding parts through integration of PSO and BP neural network
2013, Expert Systems with Applications
Citation Excerpt :
The particles’ experiences tell them to track and memorize the best locations they have ever been. For that very reason, every particle in PSO is capable of memorizing the best location it has experienced, and PSO seeks to find optimum by combining the particle best position approach and population-based search approach (Lee, 2008). In PSO, by randomly generating an initial population, the optimal solution could be found through evolutionary iterations.
Plastic injection molding technology has been widely used in a variety of high-tech products, auto parts and generic household products. Against the waves of globalization, the plastic injection enterprises must shorten time-to-market to enhancement of competence, and launch products ahead of all other competitors, and thus they can quickly seize a big target market and lead the price. The backpropagation (BP) neural network was used in this study to construct an estimating model for the cost of plastic injection molding parts so as to reduce the complexity in the traditional cost estimating procedures. Because the parameters of BP neural network have a significant influence on results, and particle swarm optimization (PSO) is capable of quickly finding optimal solutions. We integrated PSO and BP neural network so that the convergence rate was improved and precision was relatively enhanced through particle evolutions based on the optimum parameter combination from BP neural network.
Linguistic fuzzy model identification based on PSO with different length of particles
2012, Applied Soft Computing Journal
To generate the structure and parameters of fuzzy rule base automatically, a particle swarm optimization algorithm with different length of particles (DLPPSO) is proposed in the paper. The main finding of the proposed approach is that the structure and parameters of a fuzzy rule base can be generated automatically by the proposed PSO. In this method, the best fitness (f_gbest) and the number (N_gbest) of active rules of the best particle in current generation, the best fitness (f_pbesti) which ith particle has achieved so far and the number (N_pbesti) of active rules of it when the best position emerged are utilized to determine the active rules of ith particle in each generation. To increase the diversity of structure, mutation operator is used to change the number of active rules for particles. Compared with some other PSOs with different length of particles, the algorithm has good adaptive performance. To indicate the effectiveness of the give algorithm, a nonlinear function and two time series are used in the simulation experiments. Simulation results demonstrate that the proposed method can approximate the nonlinear function and forecast the time series efficiently.
Comparison and application of four versions of particle swarm optimization algorithms in the sequence optimization
2011, Expert Systems with Applications
Citation Excerpt :
Each bird, then, explores the search space from its new position, the process repeated until the flock reaches a desired destination. It is important to note that the process involves both social interaction and intelligence so the birds learn from their own experience (local search) and also from the experience of others around them (global search) (Emad et al., 2005; Lee, 2008; Yannis & Magdalene, 2010). In holes machining, the drill head moves from one hole to the next on a work-piece mounted on the CNC drill worktable.
Particle swarm optimization (PSO) algorithm is a well-known optimization approach to deal with discrete problems. There are two models proposed for the operators of PSO algorithm, one is based on value exchange and the other on order exchange, accordingly two versions of PSO algorithms are formed. A new version of PSO algorithm based on order exchange has been presented in our studies, which is capable of converging on the global optimization solution, with the method of generating the stop evolution particle over again. In this paper, we propose another version of PSO algorithm based on value exchange with the same method. There exist, thus, totally four versions of PSO algorithms, which is given a brief introduction individually and the performance of which are compared in solving sequence optimization problems through fifty runs. The performance comparison show that the PSO algorithm with global convergence characteristics based on order exchange outperforms the other versions of PSO in solving sequence optimization problem.
A novel function approximation based on robust fuzzy regression algorithm model and particle swarm optimization
2011, Applied Soft Computing Journal
In this paper, a novel approach for function approximation based on robust fuzzy regression algorithm and particle swarm optimization is proposed. First, the robust fuzzy regression algorithm is applied to construct Takagi–Sugeno–Kang fuzzy model. The robust fuzzy regression algorithm is not only to simultaneously identify parameters in the premise parts and the consequent parts, but it also defines the number of fuzzy rules to fit for Takagi–Sugeno–Kang model. In addition, the robust fuzzy regression algorithm has robust learning effects when noise and outliers exist. Thereafter, particle swarm optimization is conducted to fine tune parameters from obtained fuzzy model. In simulation results, particle swarm optimization can improve Takagi–Sugeno–Kang fuzzy model built by robust fuzzy regression algorithm efficiently. The proposed approach can find best solutions when compared with other learning algorithms for four test functions.
Parameter optimization of sub-35 nm contact-hole fabrication using particle swarm optimization approach
2010, Expert Systems with Applications
The major research focus on integrated circuits (ICs) mainly deals with increasing circuit performance and functional complexity of circuit. The lithography process is the most critical step in the fabrication of nanostructure for integrated circuit manufacturing. The most important variable in the lithography process is the line-width or critical dimensions (CDs), which perhaps is one of the most direct impact variables on the device performance and speed. This study presents a hybrid approach combining Taguchi’s robust design, back-propagation neural network modeling technique and particle swarm optimization (PSO) for sub-35 nm contact-hole fabrication in the lithography process. The BP neural network is employed to model the functional relationship between the input parameters and target responses. Particle swarm optimization is adopted to optimize the parameter settings through the well-trained BP model, where each particle is assessed using fitness function. The proposed PSO algorithm applies the velocity updating and position updating formulas to the population composed of many particles such that better particles are generated. Compared with realistic fabricated and measured data, this approach can achieve the optimal parameter settings for minimized CDs or target CDs. Meanwhile, it reduces the CD variation through the design of experiment. The experimental results show that the proposed approach dealing with the process modeling and parameter optimization demonstrates its feasibility and effectiveness for sub-35 nm contact-hole fabrication.

View all citing articles on Scopus

View full text

A novel hybrid algorithm for function approximation

Abstract

Introduction

Section snippets

The concept of Takagi and Sugeno’s fuzzy model

The introduction of genetic algorithm and particle swarm optimization

The proposed algorithm

Simulation results

Conclusion

Fuzzy Sets and Systems

Fuzzy Sets and Systems

Applied Soft Computing

Fuzzy Sets and Systems

Fuzzy Sets and Systems

Fuzzy Sets and Systems

Pattern recognition with fuzzy objective function algorithm

Robust TSK fuzzy modeling for function approximation with outliers

IEEE Transactions on Fuzzy Systems

Handling multiple objectives with particle swarm optimization

IEEE Transactions on Evolutionary Computation

Robust clustering methods: a unified view

IEEE Transactions on Fuzzy Systems

Fuzzy function approximation with ellipsoidal rules

IEEE Transactions on Systems Man and Cybernetics

A robust competitive clustering algorithm with applications in computer vision

IEEE Transactions on Pattern Analysis and Machine Intelligence

Genitic algorithms and engineering design

Genetic algorithms in search, optimization, and machine learning