Genetic optimization-driven multi-layer hybrid fuzzy neural networks

doi:10.1016/j.simpat.2005.10.009

Simulation Modelling Practice and Theory

Volume 14, Issue 5, July 2006, Pages 597-613

https://doi.org/10.1016/j.simpat.2005.10.009 Get rights and content

Abstract

In this study, we introduce a new architecture of hybrid fuzzy neural networks (gHFNNs) and offer a comprehensive design methodology that supports their development. The gHFNN rule-based architecture results from a synergistic usage of Fuzzy Neural Networks (FNNs) with Polynomial Neural Networks (PNNs). The FNN contributes to the formation of the premise part of the overall network of the gHFNN. The consequence part of the gHFNN is designed taking advantage of PNNs. The optimization of the FNN is realized with the aid of a standard back-propagation learning combined with genetic optimization. The development of the PNN dwells on the extended Group Method of Data Handling (GMDH) and Genetic Algorithms (GAs). Through the consecutive process of such structural and parametric optimization, an optimized topology of the PNN becomes generated in a dynamic fashion. The performance of the gHFNN is evaluated through a series of numeric experiments. A comparative analysis shows that the proposed gHFNN is characterized by higher accuracy as well as significant predictive capabilities when contrasted with other neurofuzzy models presented in the literature.

Section snippets

Introductory remarks

With the continuously growing demand for models for coping with complex systems that are inherently associated with nonlinearity, high-order dynamics, time-varying behavior and imprecise measurements, there is a need for a relevant modeling environment. Efficient modeling techniques should allow for a selection of pertinent variables and a formation of highly representative datasets. The models should be able to take advantage of the existing domain knowledge (such as a prior experience of

Conventional hybrid fuzzy neural networks

The architectures of conventional hybrid fuzzy neural networks (HFNN) [12], [13] result as a synergy between two general constructs such as FNN and PNN. Based on the different PNN topologies, in the HFNNs we distinguish between two kinds of architectures, see Table 1. We will be referring to them as basic and modified ones. Moreover, for the each of these architectures we identify two cases. Considering the connection point, if the number of the input variables of the PNN used in the

The architecture and the development of genetically optimized HFNNs (gHFNNS)

The genetically optimized HFNNs (gHFNN) emerges as a result of a genetically optimized multi-layer perceptron architecture based on fuzzy set-based FNN and the use of GAs and the GMDH method. In this sense, these networks come as a result of synergy between FNNs [14], [15] and PNNs [10].

An overall design procedure of the gHFNNs

Now, we concentrate on the optimization flow of the algorithm as well as link it to the detailed architectural developments of the network.

Experimental studies

The performance of the gHFNN is illustrated with the aid of a time series of gas furnace Box–Jenkins data [16]. The crossover rate of GAs is set to 0.75 and the probability of mutation is equal to 0.065. This time series (296 input–output pairs) has been intensively studied in the previous literature [1], [2], [3], [4], [5], [6], [12], [13], [14], [15], [16]. The delayed terms of methane gas flow rate, u(t) and carbon dioxide density, y(t) are used as system input variables. We use two types of

Concluding remarks

This comprehensive design methodology has resulted in parametrically as well as structurally optimized networks. Regarding the premise structure of the gHFNNs, the optimization of the rule-based FNN hinges here on genetic optimization and the back-propagation learning. GA leads to the auto-tuning of the parameters of the membership functions, while the BP produces the optimal parameters of the consequent polynomials of the rules. The gPNN that forms the consequent structure of the gHFNN

Acknowledgement

This work was supported by Korea Research Foundation Grant (KRF-2004-002-D00257).

References (21)

W. Pedrycz
An identification algorithm in fuzzy relational system
Fuzzy Set. Syst.
(1984)
S.-K. Oh et al.
Fuzzy identification by means of auto-tuning algorithm and its application to nonlinear systems
Fuzzy Set. Syst.
(2000)
S.-K. Oh et al.
Hybrid identification in fuzzy-neural networks
Fuzzy Set. Syst.
(2003)
E. Kim et al.
A simply identified Sugeno-type fuzzy model via double clustering
Inform. Sciences
(1998)
S.-K. Oh et al.
Rule-based multi-FNN identification with the aid of evolutionary fuzzy granulation
J. Knowledge-Based Syst.
(2004)
C.W. Xu et al.
Fuzzy model identification self-learning for dynamic system
IEEE Trans. Syst. Man Cyb.
(1987)
M. Sugeno et al.
A fuzzy-logic-based approach to qualitative modeling
IEEE Trans. Fuzzy Syst.
(1993)
B.-J. Park et al.
Identification of fuzzy models with the aid of evolutionary data granulation
IEE Proc-Contr. Theor. Appl.
(2001)
S.-K. Oh et al.
Hybrid identification of fuzzy rule-based models
Int. J. Intell. Syst.
(2002)
K.S. Narendra et al.
Gradient methods for the optimization of dynamical systems containing neural networks
IEEE Trans. Neural Networ.
(1991)

There are more references available in the full text version of this article.

Cited by (18)

Artificial neural networking model for the prediction of high efficiency boiler steam generation and distribution
2015, Simulation Modelling Practice and Theory
Development of artificial neural network (ANN) models using real plant data for the prediction of fresh steam properties from a brown coal-fired boiler of a Slovenian power plant. The power plant generates electrical and thermal energy used for the city-wide district heating. The energy is produced in three blocks. Each block consists of a coal-fired boiler and an extraction condensing steam turbine. The electricity production is planned, while the generation of heat for heating purposes depends on the ambient temperature. A model will be presented which, using an ANN, predicts the power production of the power plant and distributes the production between the boilers so that the latter operate at their highest efficiency. The real data on the amount of the generated steam in the existing system boilers will be compared to the results of the model and the findings will be indicated regarding the coal consumption savings and their impact on the environment. However, the final set of input parameters was optimised with a compromise between smaller number of parameters and higher level of accuracy through sensitivity analysis. Data for training were carefully selected from the available real plant data.
Artificial neural networking and fuzzy logic exergy controlling model of combined heat and power system in thermal power plant
2015, Energy
This paper presents entropy generation minimisation model of combined heat and power system. The turbine control valves and heater throttle valves were analysed. The high-pressure control valves regulate the mass flow rate of steam into the turbine, whereas the intermediate-pressure and low-pressure control valves the steam pressure of the turbine extracts 3 and 5. The steam of the turbine extracts 3 and 5 is used for the city-wide heating system purposes by means of the peak and basic heaters. The quantity of the extracted steam used for the city-wide heating system is additionally controlled by the throttles regulating the extracted steam into the basic or peak heater. This results in a double throttling of the extracted steam of the turbine, double generated entropy and a double loss of work. If adequate pressure of the extracted steam of the turbines is maintained by means of the turbine control valves the two heaters for the heating system could operate with the throttles open. As a result, the generated entropy of the throttles of the steam admitted to the heater could be avoided and the amount of generated entropy of the turbine control valves reduced.
GMDH to predict scour depth around a pier in cohesive soils
2013, Applied Ocean Research
This study presents new application of group method of data handling (GMDH) to predict scour depth around a vertical pier in cohesive soils. Quadratic polynomial was used to develop GMDH network. Back propagation algorithm has been utilized to adjust weighting coefficients of GMDH polynomial thorough trial and error method. Parameters such as initial water content, shear strength, compaction of cohesive bed materials, clay content of cohesive soils, and flow conditions are main factors affecting cohesive scour. Performances of the GMDH network were compared with those obtained using several traditional equations. The results indicated that the proposed GMDH-BP has produced quite better scour depth prediction than those obtained using traditional equations. To assign the most significant parameter on scour process in cohesive soils, sensitivity analysis was performed for the GMDH-BP network and the results showed that clay percentage was the most effective parameter on scour depth. The error parameter for three classes of IWC and C_p showed that the GMDH-BP model yielded better scour prediction in ranges of IWC = 36.3–42.28% and C_p = 35–100%. In particular application, the GMDH network was proved very successful compared to traditional equations. The GMDH network was presented as a new soft computing technique for the scour depth prediction around bridge pier in cohesive bed materials.
Neural network design and model reduction approach for black box nonlinear system identification with reduced number of parameters
2013, Neurocomputing
Citation Excerpt :
However, these methods still suffer from slow convergence [31]. More recently, evolutionary techniques as genetic algorithms (GAs) and particle swarm optimization (PSO) have been employed to optimize weights, layers, number of input–output nodes, neurons and derive optimal neural network structures [31,34]. Ge et al. [35] proposed a learning algorithm centered on a dissimilation particle swarm optimization, serving to compute the optimal synaptic weights, the learning rate, and the architecture evolution (optimal number of neurons in the hidden layer).
In this paper a dedicated recurrent neural network design and a model reduction approach are proposed in order to improve the balance between complexity and quality of black box nonlinear system identification models. The proposed neural network design, based on a three-layers architecture, helps to reduce the number of parameters of the model after the training phase without any significant loss of estimation accuracy. Nevertheless, the proposed architecture remains sufficiently general to provide a wide range of models among the most encountered in the literature. This reduction, achieved by a convenient choice of the activation functions and the initial conditions of the synaptic weights, is developed in two steps. The first step is to train the proposed architecture under two reasonable assumptions. Then the recurrent three-layers neural network is transformed into a representation of two-layer with less number of neurons, that is, a significant reduced number of parameters. The constructed architecture provided models with reasonable reduced number of parameters with a convenient estimation accuracy. To validate the proposed approach, we identify the Wiener–Hammerstein benchmark nonlinear system proposed in SYSID2009 [1].
Genetic algorithms based logic-driven fuzzy neural networks for stability assessment of rubble-mound breakwaters
2012, Applied Ocean Research
Citation Excerpt :
They concluded that the presented type of FNNs needed to improve its computational performance, since the maximum value of the correlation coefficient between predicted stability numbers and measurements from the hydraulic model tests [19], was limited to 0.788. One common way to overcome this problem in such networks is considering a hybrid learning scheme in which structural and parametric optimisation are implemented separately [20,21]. Structural optimisation generally aims to construct a blueprint of the network by discovering the most meaningful connections that shape up an underlying architecture which reflects the logical nature of the data.
This study focuses on the further development of fuzzy neural network (‘FNN’) models for the prediction of stability numbers for the design of rubble mound breakwaters. It introduces two new FNN models namely: (i) the genetic algorithm-based fuzzy neural network (‘GA-FNN’); and (ii) the hybrid genetic algorithm-based fuzzy neural network (‘HGA-FNN’). GA-FNN uses a standard genetic algorithm (‘GA’) to optimise both its structure and parameters. HGA-FNN is the extension of GA-FNN; however, a conditional local search method is involved. The results show that HGA-FNN has a better predictive performance than GA-FNN and that it has good potential in terms of stability assessments of coastal structures.
A PSO-based adaptive fuzzy PID-controllers
2012, Simulation Modelling Practice and Theory
In this paper, a novel design method for determining the optimal fuzzy PID-controller parameters of active automobile suspension system using the particle swarm optimization (PSO) reinforcement evolutionary algorithm is presented. This paper demonstrated in detail how to help the PSO with Q-learning cooperation method to search efficiently the optimal fuzzy-PID controller parameters of a suspension system. The design of a fuzzy system can be formulated as a search problem in high-dimensional space where each point represents a rule set, membership functions, and the corresponding system’s behavior. In order to avoid obtaining the local optimum solution, we adopted a pure PSO global exploration method to search fuzzy-PID parameter. Later this paper explored the improved the limitation between suspension and tire deflection in active automobile suspension system with nonlinearity, which needs to be solved ride comfort and road holding ability problems, and so on. These studies presented many ideas to solve these existing problems, but they need much evolution time to obtain the solution. Motivated by above discussions this paper propose a novel algorithm which can decrease the number of evolution generation, and can also evolve the fuzzy system for obtaining a better performance.

View all citing articles on Scopus

View full text

Genetic optimization-driven multi-layer hybrid fuzzy neural networks

Abstract

Section snippets

Introductory remarks

Conventional hybrid fuzzy neural networks

The architecture and the development of genetically optimized HFNNs (gHFNNS)

An overall design procedure of the gHFNNs

Experimental studies

Concluding remarks

Acknowledgement

Fuzzy Set. Syst.

Fuzzy Set. Syst.

Fuzzy Set. Syst.

Inform. Sciences

J. Knowledge-Based Syst.

Fuzzy model identification self-learning for dynamic system

IEEE Trans. Syst. Man Cyb.

A fuzzy-logic-based approach to qualitative modeling

IEEE Trans. Fuzzy Syst.

Identification of fuzzy models with the aid of evolutionary data granulation

IEE Proc-Contr. Theor. Appl.

Hybrid identification of fuzzy rule-based models

Int. J. Intell. Syst.

Gradient methods for the optimization of dynamical systems containing neural networks

IEEE Trans. Neural Networ.