Generalized type-2 fuzzy weight adjustment for backpropagation neural networks in time series prediction

doi:10.1016/j.ins.2015.07.020

Information Sciences

Volume 325, 20 December 2015, Pages 159-174

https://doi.org/10.1016/j.ins.2015.07.020 Get rights and content

Abstract

In this paper the comparison of a proposed neural network with generalized type-2 fuzzy weights (NNGT2FW) with respect to the monolithic neural network (NN) and the neural network with interval type-2 fuzzy weights (NNIT2FW) is presented. Generalized type-2 fuzzy inference systems are used to obtain the generalized type-2 fuzzy weights and are designed by a strategy of increasing and decreasing an epsilon variable for obtaining the different sizes of the footprint of uncertainty (FOU) for the generalized membership functions. The proposed method is based on recent approaches that handle weight adaptation using type-1 and type-2 fuzzy logic. The approach is applied to the prediction of the Mackey–Glass time series, and results are shown to outperform the results produced by other neural models. Gaussian noise was applied to the test data of the Mackey–Glass time series for finding out which of the presented methods in this paper shows better performance and tolerance to noise.

Introduction

One of the most important parts in the structure of the neural network can be considered to be the weights applied to the neurons, because these enable the learning process in the neural network. This part is the main focus of this paper, because the use of generalized type-2 fuzzy weights to obtain the values for the weights in the connections between the input and hidden layers, and the hidden and output layers for a neural network can improve the learning process. The proposed method is compared against a neural network with interval type-2 fuzzy weights using the same architecture and learning algorithms. Noise was also applied to the real data to analyze the performance of the methods under a higher level of uncertainty.

We propose the adaptation of the weights in the backpropagation algorithm for neural networks using generalized type-2 fuzzy inference systems [2]. This approach is different than the ones previously used in the literature, where the adaptation of the weights is made with a momentum variable and adaptive learning rate [4], [20], or using triangular or trapezoidal type-1 fuzzy numbers to describe the weights [28], [29]. Also, in previous work we presented interval type-2 fuzzy inference with triangular or Gaussians membership functions to obtain the weights for the neurons in the neural network [21], [22].

The proposed approach is applied to time series prediction for the Mackey–Glass time series. The objective of applying different forecasting models is obtaining the minimum prediction error for the data of the time series.

The paper is mainly based on comparing the performance for the neural network with generalized type-2 fuzzy weights against a neural network with interval type-2 fuzzy weights. This is an important issue to investigate; because the weights affect the learning process of the neural network and considering uncertainty in the weights of the neural network allow obtaining better results. We performed experiments with the two methods and presented the results and comparison of these methods for the prediction of the Mackey–Glass time series.

The main contribution is the proposed adaptation of the backpropagation algorithm to allow the neural network to manage data with uncertainty, using generalized type-2 fuzzy logic for obtaining generalized type-2 fuzzy weights in the connections between the neurons of the layers. This adaptation of the backpropagation algorithm using generalized type-2 fuzzy inference systems to generate the type-2 weights enables the neural network to handle data with noise, such as that of the Mackey–Glass time series and other complex time series.

The paper is structured as follows: the next section presents a background of different methods for managing the weights and modifications of the backpropagation algorithm in neural networks, and basic concepts of neural networks and generalized and interval type-2 fuzzy logic. Section 3 explains the proposed method and the problem description, the weights using generalized type-2 fuzzy systems, and the neural network architecture with generalized type-2 fuzzy weights proposed in this paper. Section 4 presents the simulation results for the proposed methods. Finally, in Section 5 the conclusions of this work are presented.

Section snippets

Neural networks

An artificial neural network is based on the processing of artificial neurons connections [14], [37]. The artificial neuron is composed of several elements as described below.

First we find the inputs and the weights for each input, these are connected to the neuron and then the neuron performs a summation of the multiplication of the input values for the weights ( $\sum^{x_{i}} w_{i j}$ ), and finally the activation function is used. This function is completed with the addition of a threshold amount i. This

Proposed method and problem description

The proposed approach presented in this paper is an enhancement of the traditional backpropagation algorithm in which we are now using generalized type-2 fuzzy sets to allow the neural network to handle data with uncertainty. In generalized type-2 fuzzy sets is necessary to systematically vary the footprint of uncertainty (FOU) of the membership functions to properly design the fuzzy systems for the corresponding applications.

The proposed method utilizes generalized type-2 fuzzy weights in the

Simulation results

The results obtained in the experiments using the neural network with generalized type-2 fuzzy weights are presented in Table 1 and Fig. 9. In this case all parameters of the neural network are established empirically. The best achieved prediction error is of 0.0548, and the average error is of 0.0714. We are presenting 10 experiments in Table 1, but the average error is calculated considering 40 experiments with the same parameters and conditions. The number of epochs for the network is of 100

Conclusions

In the experiments, when there is no noise present, we observe that the neural network with interval type-2 fuzzy weights obtains better results than the neural network with generalized type-2 fuzzy weights for the Mackey–Glass time series. However, the neural network with generalized type-2 fuzzy weights is better at different noise levels than the neural network with interval type-2 fuzzy weights and the monolithic neural network. This conclusion is based on the fact that for different levels

References (61)

AlievR.A. et al.
Type-2 fuzzy neural networks with fuzzy clustering and differential evolution optimization
Inf. Sci.
(2011)
BarbounisT.G. et al.
Locally recurrent neural networks for wind speed prediction using spatial correlation
Inf. Sci.
(2007)
CasasentD. et al.
A classifier neural net with complex-valued weights and square-law nonlinearities
Neural Netw.
(1995)
CastilloO. et al.
A review on the design and optimization of interval type-2 fuzzy controllers
Appl. Soft Comput.
(2012)
CastroJ. et al.
A hybrid learning algorithm for a class of interval type-2 fuzzy neural networks
Inf. Sci.
(2009)
DraghiciS.
On the capabilities of neural networks using limited precision weights
Neural Netw.
(2002)
Fazel ZarandiM.H. et al.
A new indirect approach to the type-2 fuzzy systems modeling and design
Inf. Sci.
(2013)
GaxiolaF. et al.
Interval type-2 fuzzy weight adjustment for backpropagation neural networks with application in time series prediction
Inf. Sci.
(2014)
GedeonT.
Additive neural networks and periodic patterns
Neural Netw.
(1999)
GreenfieldS. et al.
Defuzzification of the discretised generalised type-2 fuzzy set: experimental evaluation
Inf. Sci.
(2013)

CastilloO. et al.

(2001)

CastilloO. et al.

Type-2 Fuzzy Logic Theory and Applications

(2008)

CastroJ. et al.

An interval type-2 fuzzy logic toolbox for control applications

CastroJ. et al.

An interval type-2 fuzzy neural network for chaotic time series prediction with cross-validation and akaike test

Soft Computing for Intelligent Control and Mobile Robotics

(2011)

CastroJ. et al.

Building fuzzy inference systems with a new interval type-2 fuzzy logic toolbox

Trans. Comput. Sci.

(2008)

Cited by (55)

Design and prediction of self-organizing interval type-2 fuzzy wavelet neural network
2024, Information Sciences
This paper presents a self-organizing interval type-2 fuzzy wavelet neural network (SIT2FWNN) model for predicting and identifying nonlinear systems. Based on traditional TSK interval type-2 fuzzy neural network (IT2FNN), the proposed SIT2FWNN utilizes the wavelet basis function with the locating abilities of time-domain and frequency-domain as the consequent of fuzzy rules, combining the capacity of IT2FNN to handle the uncertainty and the great learning potentiality of wavelet neural network (WNN). For the structural adjustment of SIT2FWNN, fuzzy rules are added and deleted according to the Euclidean distance between the input layer and the fuzzification layer. The self-organizing algorithm can delete redundant and unimportant fuzzy rules, thus optimizing the structure of SIT2FWNN and simplifying the calculation. In parameter learning, the AdaBound algorithm and self-adaptive gradient learning algorithm are used to find optimal values of unknown parameters of the SIT2FWNN model. Finally, the designed SIT2FWNN model is applied in the predictions of short-term traffic flow, Mackey-Glass time series, and the opening index of the Shanghai stock index. The evaluation comparison between the proposed model and similar studies proves that SIT2FWNN has higher prediction accuracy and speed.
Downhole parameter prediction method based on multi-layer water injection model and historical data-based model parameter identification
2023, Heliyon
Wireless communication has become a preferred direction for the development of layered water injection tools due to its low cost and high reliability. However, the wireless system relies on the underground battery for power supply，and each communication will consume a significant amount of energy. In order to save energy consumption, the wireless system adopts the intermittent sleep communication mode, with intervals of usually more than one month. During the idle time of communication, the downhole parameters such as pressure and flowrate will change as the pressure and flowrate at the wellhead. Therefore, it is crucial to predict downhole parameters based on the wellhead pressure and flowrate. In this study, a downhole parameter prediction method based on multi-layer water injection model is proposed. A multilayer injection prediction model was established based on the hydraulic analysis of the tubing string, and the model parameters were identified and updated using the historical data uploaded each time. The pressure and flow rate measured at the wellhead were used as inputs to the model, and the recursive relationship between layers in the multilayer model was utilized to predict downhole parameters for each layer. A model parameter optimization method based on time-weighting is proposed in order to address the gradual changes in model parameters during water injection. This method assigns greater weight to more recent historical data, resulting in optimized model parameters. Experimental results show that the proposed method can effectively predict the flowrate and pressure of each layer, with a prediction deviation of less than 5% F.S., which provides technical support for the application and popularization of the wireless layered water injection system.
Explainability techniques applied to road traffic forecasting using Graph Neural Network models
2023, Information Sciences
In recent years, several new Artificial Intelligence methods have been developed to make models more explainable and interpretable. The techniques essentially deal with the implementation of transparency and traceability of black box machine learning methods. Black box refers to the inability to explain why the model turns the input into the output, which may be problematic in some fields. To overcome this problem, our approach provides a comprehensive combination of predictive and explainability techniques. Firstly, we compared statistical regression, classic machine learning and deep learning models, reaching the conclusion that models based on deep learning exhibit greater accuracy. Of the great variety of deep learning models, the best predictive model in spatio-temporal traffic datasets was found to be the Adaptive Graph Convolutional Recurrent Network. Regarding the explainability technique, GraphMask shows a notably higher fidelity metric than other methods. The integration of both techniques was tested by means of experimental results, concluding that our approach improves deep learning model accuracy, making such models more transparent and interpretable. It allows us to discard up to 95% of the nodes used, facilitating an analysis of its behavior and thus improving the understanding of the model.
Neural network operators of fuzzy n-cell number valued functions and multidimensional fuzzy inference system
2022, Knowledge-Based Systems
In this paper, we introduce a novel family of neural network operators of fuzzy n-cell number valued functions, activated by a collection of multivariate sigmoidal functions. We give some special examples of these activation functions with graphs and present some illustrative examples to demonstrate the approximation performance of these operators. Moreover, we propose a multidimensional fuzzy inference system including neural network operators of fuzzy n-cell number valued functions for the symptom-based diagnosis of Covid-19 disease. Finally, we give some approximation results using an $L_{p}$ type metric of fuzzy n-cell numbers and examine the rate of convergence for the operators by means of fuzzy $L_{p}$ modulus of continuity.
Multivariate fuzzy neural network interpolation operators and applications to image processing
2022, Expert Systems with Applications
In this paper, we introduce a novel family of multivariate fuzzy neural network interpolation operators activated by sigmoidal functions belonging to the new class of multivariate sigmoidal functions. To present an alternative way to the well-known shortcomings of the Hukuhara difference, we use a proper function defined on a set of fuzzy $n$ -cell numbers. Moreover, we construct the Kantorovich variant of fuzzy NN interpolation operators, and also achieve approximation properties via $L^{p}$ -type metric with respect to both modulus of continuity and $L^{p}$ -modulus of continuity in fuzzy sense. Various special examples for the class of multivariate sigmoidal functions are presented. Also, we give some illustrative examples to demonstrate the approximation performances of all the above operators. Finally, we give a novel interpolation algorithm involving a multidimensional fuzzy inference system with applications in color image resizing and inpainting.
Convergent newton method and neural network for the electric energy usage prediction
2022, Information Sciences
In the neural network adaptation, the Newton method could find a minimum with its second-order partial derivatives, and convergent gradient steepest descent could assure its error convergence with its time-varying adaptation rates. In this article, the convergent Newton method is proposed as the combination of the Newton method and the convergent gradient steepest descent for the neural networks adaptation, where the convergent Newton method incorporates the second-order partial derivatives inside of the time-varying adaptation rates. Hence, the convergent Newton method could assure its error convergence and could find a minimum. Experiments show that the convergent Newton method obtains satisfactory results in the electric energy usage data prediction.

View all citing articles on Scopus

View full text

Generalized type-2 fuzzy weight adjustment for backpropagation neural networks in time series prediction

Abstract

Introduction

Section snippets

Neural networks

Proposed method and problem description

Simulation results

Conclusions

Inf. Sci.

Inf. Sci.

Neural Netw.

Appl. Soft Comput.

Inf. Sci.

Neural Netw.

Inf. Sci.

Inf. Sci.

Neural Netw.

Inf. Sci.

Expert Syst. Appl.

Neural Netw.

Inf. Sci.

Inf. Sci.

Neural Netw.

Inf. Sci.

Neural Netw.

Neural Netw.

Inf. Sci.

Neurocomputing

Inf. Sci.

Inf. Sci.

A type-2 fuzzy wavelet neural network for time series prediction

Lecture Notes in Computer Science

A derivation of conjugate gradients

Numerical Methods for Nonlinear Optimization

Multiobjective optimization and comparison of nonsingleton type-1 and singleton interval type-2 fuzzy logic systems

IEEE Trans. Fuzzy Syst.

Type-2 Fuzzy Logic Theory and Applications

An interval type-2 fuzzy logic toolbox for control applications

An interval type-2 fuzzy neural network for chaotic time series prediction with cross-validation and akaike test

Soft Computing for Intelligent Control and Mobile Robotics

Building fuzzy inference systems with a new interval type-2 fuzzy logic toolbox

Trans. Comput. Sci.