A Beta basis function Interval Type-2 Fuzzy Neural Network for time series applications
Introduction
Fuzzy neural networks have been widely used in intelligent methodologies to settle serious data science problems, since they provide better learning capabilities. For instance, they have been fruitful applied in solving non linear and complex systems. Fuzzy systems (FSs) have been demonstrated to have good approximation capabilities (Wang, 1992), which have been used widely for approximating non linear functions and behaviors and forecasting many activities. Regarding learning type-1 fuzzy systems applied to regression, non-linear identification and time series problems, several works have been considered. In Fletcher and Reeves (1964), The neural network weights adaptation is proposed using an adaptive learning rate and momentum variable. An adaptive neural fuzzy inference system have been proposed first by Jang in 1998 (Jang, 1993) denoted and known by ANFIS. This Neural Fuzzy System (NFS) is based on an adaptive neural network structure using either the backpropagation algorithm or hybrid learning algorithm and a TSK model fuzzy logic system. In 1997, Mendel elaborated a type-1 FLS for the first time applied to the Mackey glass time series prediction problem (Mendel and Mouzouris, 1997). The TSK fuzzy structure was considered in several studies. In Jang (1993), Jang has defined an adaptive neural fuzzy inference system applied to the chaotic time series problem identification. For the learning process the gradient descent based approach was utilized. In Juang and Lin (1998), a SONFIN self constructing type-1 TSK FNN having an online learning ability was suggested using a competitive learning method. In Wu and Er (2000), a hierarchical on-line fuzzy neural network based radial basis function for TSK systems was proposed. Then through many theoretical studies and several successful applications, type-2 fuzzy neural systems have proved their effectiveness regarding type-1 FNNs. In 1999 (Karnik and Mendel, 1999), T2 concepts were added and applied for the Mackey glass time series prediction problem. In Liang and Mendel (2000), Liang and Mendel proposed a simplification of the generalized type-2 fuzzy set theory to introduce by this the concept of interval fuzzy sets. Since, much intention has been dutiful to learning type-2 FLS to have better approximation of non linear aptitudes Das et al. (2015), Tung et al. (2013), Lee et al. (2003), Wang et al. (2004), Juang and Tsao (2008). Jung in Juang and Tsao (2008), proposed an online structure of a TSK type-2 fuzzy neural network for fuzzy parameter learning. In this study, the antecedent fuzzy parameters were tuned using gradient descent algorithm while consequent parameters were tuned using a kalman filter algorithm. While in Juang et al. (2010), an interval type-2 fuzzy neural network (IT2FNN) combined with support vector regression structure (IT2FNN-SVR) was proposed. In Mndez and de los Angeles Hernandez (2009), a hybrid learning methodology has been developed for interval FNN. The back-propagation algorithm and the recursive orthogonal least squares algorithm, were both applied for adjusting respectively antecedent and consequent membership functions fuzzy parameters. In MaNdez and De Los Angeles Hernández (2013), Mendez proposed a hybrid learning algorithm for settling parameters of a non-singleton interval A2-C1 model type-2 TSK FLS system. In Das et al. (2015) a sequential meta-cognitive architecture of a learning algorithm has been introduced using a model of a TSK type-2 NFS. Generally, the performance of a neural network relies essentially on two matters to know the network architecture and the transfer function used in layers. Regarding the network structure the most used and simplest in neural fuzzy architectures is the feedforward network with a simple or hybrid backpropagation weight adjusting algorithm. In learning type-2 fuzzy neural networks, the backpropagation algorithm with its variations is the most used weight adjusting method for tuning fuzzy neural networks parameters. This algorithm consists on reducing through iterations, error between actual (FS) output network and a desired output. The initial use of this algorithm for a fuzzy system was in 1992 (Wang and Mendel, 1992), since, this algorithm have been proved its efficiency in several fuzzy neural leaning process. The backpropagation algorithm was defined for TSK interval type-2 FLS first in Mendel (2001a). Derivatives and formulas that are needed to accomplish this algorithm were provided in Mendel (2004). Those mathematical relations have been given for only gaussian membership functions with uncertain standard deviations or uncertain mean as in Rubio-Solis and Panoutsos (2015). Learning type-2 FLS parameters have been proposed at first in Lee et al. (2003), Mendel (2004), Wang et al. (2004) and Uncu and Turksen (2007). In Castro et al. (2009), three (IT2FNN)s structures were defined using gradient descent backpropagation with and without adaptive learning procedure. While in Wang et al. (2004) an IT2FNN was proposed using backpropagation algorithm. In Gaxiola et al. (2014b) a backpropagation leaning algorithm is used in the FNN with type-1 and type-2 triangular fuzzy weights. In Gaxiola et al. (2015), authors proposed a neural network with a generalized type-2 fuzzy weights denoted (NNGT2FW) and presented a comparison to the neural network with interval type-2 fuzzy weights (NNIT2FW). Analysis were ensured using prediction of Mackey Glass time series. In the considered neural network back-propagation algorithm is applied and the adaptation of its weight is ensured using generalized type-2 fuzzy inference systems. Regarding the transfer function used in layers, for fuzzy neural networks we mean the shape of membership function used for defining a fuzzy system. As it was noticed in Wang et al. (2004), since the variation of initial values of membership functions may effect the performance of the training process, by consequent also the used kind of shapes of membership function may also alter the performance result of training. The most considered membership function in literature is the gaussian membership function (Gaxiola et al., 2014a). But, this does not exclude the existence of triangular and trapezoidal membership functions in some works Ishibuchi et al. (1993), Ishibuchi et al. (1995). In the context of comparing membership functions (MFs), in Olivas et al. (2014) authors presented a comparative study on the impact of the utilization of triangular and gaussian MFs in an IT2 fuzzy system for adjusting the Particle Swarm Optimization algorithm parameters. Considering type-2 fuzzy systems (T2FSs), the major cause of migration from type-1 (T1) to type-2 (T2) systems is particularly in the reduction of error and uncertainty that could be provided by T2FSs. But, the contradiction that arises here is : on the one hand, type-2 fuzzy systems which thanks to their type-2 membership functions that afforded the opportunity to handle uncertainties, but, on the other hand, almost all studies about, have neglected the possible impact of the chosen shape of the type-2 membership function on the further reduce of that uncertainty. Nevertheless it was mentioned in an earliest work (Wang, 1992) the possible impact of the shape of chosen membership function of a fuzzy system on the smoothness of the input–output surface. In this context, we demonstrated in this paper the great effect that can yield a rich membership function, such the beta function. This function was firstly proposed by Alimi in 1997 (Alimi, 1997a), as a transfer function in an artificial feedforward three layers neural network denoted the Beta Basis Function Neural Network (BBFNN). The beta function has several benefits comparing to the gaussian one, for instance it has the ability to provide more rich shapes specifically in points of view linearity, asymmetry and flexibility Alimi et al. (2000), Alimi (2000), Alimi et al. (2003). Successful applications have been utilized this function including classification and pattern recognition Alimi (1997b), Ltaief et al. (2012), Bezine et al. (2007), Bezine et al. (2003), modeling of neural networks Bouaziz et al. (2013), Bouaziz et al. (2014), and time series forecasting (Baklouti et al., 2015). In this paper, a Beta basis function Interval Type-2 Fuzzy Neural Network architecture BIT2FNN was introduced. Throughout this architecture, an interval type-2 beta fuzzy set was defined and first order derivatives of both type-1 and type-2 beta sets were calculated. Based on a given data pairs and upon the backpropagation learning algorithm, weight adjusting and fuzzy parameters update of the antecedent and consequent part were performed. Comparison results were carried out using both beta and gaussian membership functions. The BIT2FNN model was tested using essentially two examples of time series applications: the Mackey Glass Chaotic Time-Series prediction application with different setting of parameters and levels of noise and the ECG heart-rate Time Series monitoring application. The obtained results with beta functions presented good performances. The paper is organized by following; Section 2 presented a description of the interval type-2 beta fuzzy set and properties including the beta primary MF with uncertain center. Next section, considered calculation details of first order derivatives of type-1 and type-2 beta functions. In Section 4 the structure of the BIT2FNN was detailed. Then it follows a description of the used backpropagation learning methodology. In Section 6, simulation studies are depicted. Finally the paper is concluded.
Section snippets
Type-1 beta fuzzy set
A type-1 beta fuzzy set in one dimensional case is the earlier defined beta membership function in Alimi (1997a) and expressed as follows, and are real values with and . The beta function center is defined by and its width by . The great significance and worth of the beta function relies mainly on its ability to approximate several functions such as triangular and gaussian
First order derivatives of type-2 beta functions
Since our research study in this paper is the first which is defining type-2 fuzzy systems with beta membership functions, and for ensuring the training of type-1 and type-2 beta neural fuzzy systems using the back-propagation learning method, first order derivatives of type-1 and type-2 beta functions need to be elaborated.
Beta basis function Interval Type-2 Fuzzy Neural Network(BIT2FNN)
The proposed architecture in our paper consists on the design of a MISO (multi-inputs/single-output) TSK (Takagi–Sugeno–Kang) (Takagi and Sugeno, 1985) Beta Basis Function Interval Type-2 Fuzzy Neural Network, which we note BIT2FNN. This model, point of view fuzzy system relies on an antecedent part based on beta type-2 membership functions. While consequent part is type-1 fuzzy sets based outputs.
A beta-TSK fuzzy basis system is defined by the up next relation (21):
Backpropagation learning methodology
The beta fuzzy membership functions parameters are adjusted using backpropagation leaning algorithm. The error will be by then back propagated from the last layer until arriving to the first. And the antecedent and consequent parameters will be altered by the gradient descent method in accordance with the error between actual system output and the desired output. For a given input–output pairs , the main idea is to find a FLS such that the error function given in Eq. (33) is minimized:
Example 1 (a): free noise Mackey Glass Chaotic Time-Series prediction
To evaluate the performance of Beta fuzzy sets we test the well known benchmark of Forecasting of Time-Series. The considered problem is an important case study that appears in many analysis, where the main idea is better weather forecasts can, better the return on an investment can occurs. In this part we are considering a free noise data. The Mackey Glass Chaotic Time-Series data can be generated using the delay differential equation:
The setting
Conclusion
In this paper, a Beta Basis Function Interval Type-2 Fuzzy Neural Network BIT2FNN was proposed for real time series applications. Throughout this fuzzy network a type-2 beta fuzzy set was utilized. First order derivatives of type-1 and type-2 beta functions were developed and tuning parameters calculations were ensured based on a given input–output data pairs and the backpropagation learning algorithm. Simulation results have been performed by exploring two examples of time series applications:
Acknowledgment
The research leading to these results has received funding from the Ministry of Higher Education and Scientific Research of Tunisia under the grant agreement number LR11ES48. Ajith Abraham acknowledges the support from the framework of the IT4Innovations Center of Excellence project, reg. no. CZ.1.05/1.1.00/02.0070 supported by Operational Program ‘Research and Development for Innovations’ funded by Structural Funds of the European Union and state budget of the Czech Republic.
References (60)
- et al.
A hybrid learning algorithm for evolving flexible beta basis function neural tree model
Neurocomputing
(2013) - et al.
A hybrid learning algorithm for a class of interval type-2 fuzzy neural networks
Inform. Sci.
(2009) - et al.
A novel approach for classification of {ECG} arrhythmias: Type-2 fuzzy clustering neural network
Expert Syst. Appl.
(2009) - et al.
Non-singleton genetic fuzzy logic system for arrhythmias classification
Eng. Appl. Artif. Intell.
(2011) - et al.
Interval type-2 fuzzy weight adjustment for backpropagation neural networks with application in time series prediction
Inform. Sci.
(2014) - et al.
Generalized type-2 fuzzy weight adjustment for backpropagation neural networks in time series prediction
Inform. Sci.
(2015) - et al.
Applications of Type-2 fuzzy logic systems to forecasting of time-series
Inform. Sci.
(1999) - et al.
A review on the applications of type-2 fuzzy logic in classification and pattern recognition
Expert Syst. Appl.
(2013) - et al.
Ventricular fibrillation and tachycardia detection from surface {ECG} using time-frequency representation images as input dataset for machine learning
Comput. Methods Programs Biomed.
(2017) - et al.
{ECG} signal processing for abnormalities detection using multi-resolution wavelet transform and Artificial Neural Network classifier
Measurement
(2013)
Estimation of heart rate signals for mental stress assessment using neuro fuzzy technique
Appl. Soft Comput.
et2fis: An evolving type-2 neural fuzzy inference system
Inform. Sci.
{ECG} signal enhancement based on improved denoising auto-encoder
Eng. Appl. Artif. Intell.
The Beta system : toward a change in our use of neuro-fuzzy systems
Int. J. Manag.
Beta neuro-fuzzy systems
TASK Quart. J. Spec. Issue Neural Netw.
Beta fuzzy logic systems: Approximation properties in the siso case
Int. J. Appl. Math. Comput. Sci.
Beta fuzzy logic systems: Aproximation propreties in the mimo case
Int. J. Appl. Math. Comput. Sci.
On the beta-elliptic model for the control of the human arm movement
IJPRAI
An evolving interval type-2 neurofuzzy inference system and its metacognitive sequential learning algorithm
IEEE Trans. Fuzzy Syst.
Function minimization by conjugate gradients
Comput. J.
Application of adaptive neuro-fuzzy inference system for detection of electrocardiographic changes in patients with partial epilepsy using feature extraction
Expert Syst. Appl.
PhysioBank, PhysioToolkit, and PhysioNet
Circulation
Learning by fuzzified neural networks
Internat. J. Approx. Reason.
Cited by (43)
A psychometric modeling approach to fuzzy rating data
2022, Fuzzy Sets and SystemsAn optimal washout filter for motion platform using neural network and fuzzy logic
2022, Engineering Applications of Artificial IntelligenceCitation Excerpt :Di et al. (2001) proposed a highly efficient controller for an arc welding machine using FLC and NN. Baklouti et al. (2018) introduced a type-2 FNN for better approximation of a complex system with better dealing with uncertainty. Antanasijević et al. (2018) predicted the transition temperature of bent-core liquid crystals using FLC and general regression NN.
A novel probabilistic intuitionistic fuzzy set based model for high order fuzzy time series forecasting
2021, Engineering Applications of Artificial IntelligenceReview of Soft Computing Techniques for Modeling, Design, and Prediction of Wastewater Removal Performance
2021, Soft Computing Techniques in Solid Waste and Wastewater ManagementT2-FDL: A robust sparse representation method using adaptive type-2 fuzzy dictionary learning for medical image classification
2020, Expert Systems with ApplicationsCitation Excerpt :The results of this paper show the outperformance of interval type-2 fuzzy logic in comparison with type-1 fuzzy logic and Sugeno Integral in fingerprint recognition cases. Baklouti et al. applied a beta basis function interval type-2 fuzzy neural network to the consequent part of a Takagi-Sugeno-Kang (TSK) fuzzy system in chaotic time series forecasting (Baklouti, Abraham, & Alimi, 2018). By considering a new framework for active contour models, a narrow band interval type-2 fuzzy method is introduced in Shi, Lei, and Zhou (2016), which shows the efficiency and effectiveness of this type of fuzzy sets in handling the uncertainty in image segmentation tasks.