A locally recurrent fuzzy neural network with application to the wind speed prediction using spatial correlation

Abstract

In this paper, a locally feedback dynamic fuzzy neural network (LF-DFNN) for modeling of temporal processes is suggested. The model is composed of dynamic TSK-type fuzzy rules where the consequent sub-models are implemented by recurrent neural networks with internal feedback paths and dynamic neuron synapses. The LF-DFNN exhibits some interesting features, such as enhanced representation power, local modeling characteristics, model parsimony, and stable learning. Training of the LF-DFNN models is achieved using an optimal on-line learning scheme, the decoupled recursive prediction error algorithm (DRPE). The method has reduced computational demands and is derived through decomposition of the weight vector to several mutually exclusive weight groups. The partial derivatives required for the implementation of the training algorithm are calculated using the adjoint model approach, adapted to the fuzzy network's architecture exercised here. The paper deals with the wind speed prediction in wind farms, using spatial information from remote measurement stations. The LF-DFNN networks are used as advanced forecast models, providing multi-step ahead wind speed estimates from 15 min to 3 h ahead. Extensive simulation results demonstrate that our models exhibit superior performance compared to other network types suggested in the literature. Furthermore, it is shown that DRPE outperforms three gradient descent algorithms, in training of the recurrent forecast models.

Introduction

Recently, considerable research has been directed to developing recurrent fuzzy neural networks for modeling of temporal processes, control, communications and pattern recognition. Depending on the way dynamics is introduced, these models can be classified into two major categories. The former class includes models with external feedback [12], [11], [26], [32], [22], [10], while the later one includes fuzzy models with internal recurrence [17], [13], [14], [15], [18], [20]. In [12], an ANFIS structure with external feedback is used as a neuro-fuzzy controller while in [11] a recurrent fuzzy system (RFS) is suggested composed of several TSK models with crisp outputs which are interconnected through internal variables and feedback loops. A recurrent neuro-fuzzy model with multiple external feedback is suggested in [26] where the rule sub-models are linear polynomials of the inputs. The model inputs include the current external variables as well as delayed outputs of the model. A similar architecture is suggested in [32] with single-tap delay feedback and used to model a neutralization process. In [22] a dynamic fuzzy logic system (FLS) is proposed with multiple external feedback. Finally, both feedforward and recurrent NARMAX-type prediction modes are developed in [10] using a fuzzy neural network approach. An efficient algorithm is suggested for structure determination and parameter identification with the scope to improving the predictive performance of the obtained models.

With regard to the models belonging to the second class, the internal dynamics is usually introduced in the premise part of the model. In the recurrent fuzzy neural network (RFNN) suggested in [17], single-tap delay feedbacks are devised around the membership functions. A recurrent self-organizing neural fuzzy inference system (RSONFIN) is suggested in [13], [14]. An internal feedback loop is introduced in RSONFIN by circulating the firing strengths of the rules. The feedback path comprises the context nodes and the associated feedback nodes. The outputs of the context nodes provide the internal variables, used to memorize the temporal history paths of the network. The model is composed of Mamdani-type rules, implementing a dynamic fuzzy inference reasoning mechanism. Following a similar approach, TSK recurrent fuzzy network is proposed in [15]. The model is composed of dynamic TSK fuzzy rules and is used for dynamic system modeling and control. In [18] a recurrent compensatory neuro-fuzzy system is proposed where feedback connections are introduced in the term nodes, acting as memory units.

In this paper, we suggest a novel recurrent fuzzy neural network, called the locally feedback dynamic fuzzy neural network (LF-DFNN), with the model dynamics introduced in the consequent part. The model is composed of recurrent fuzzy rules of TSK type. To improve the representation capabilities of the model, the rule sub-models are implemented by means of locally feedback multi-layered perceptron networks (LF-MLP) [2], [27], including dynamic neuron units. The neuron models embedded to the consequent level are realized by linear infinite impulse response (IIR) synaptic filters. The LF-DFNN model exhibits a number of interesting qualities, such as local modeling decomposition, rich temporal representation capabilities, model parsimony with reduced complexity and stability monitoring throughout the learning process. Our approach is an extension of the NARA [25] and the CANFIS [21] models that employ static rule models, in the realm of recurrent networks. In [20] DFNN structure is presented with similar characteristics, where single-layered recurrent neural networks are considered in the consequent part, including Frasconi–Gori–Soda neurons [9], with output feedback architecture. The present work is also an extension of the above model, with regard to the following aspects. The recurrent sub-models are extended to multi-layered architectures, thus enriching the model's structure and improving its representation power. Additionally, as opposed to the batch-type learning performed in [20], we are focused on developing effective on-line learning schemes to address the wind-forecasting problem.

Training of the recurrent networks is usually accomplished through the use of gradient descent algorithms, such as the real-time recurrent learning (RTRL) [31], and the backpropagation through time (BPTT) [30]. In the RTRL method, the gradients are calculated forwards as the network runs in time, through the use of the so-called sensitivity models, while in BPTT the gradient computations are performed in a backward direction, opposite to the signal flows. To account for the complexity of the LF-DFNN suggested here, we resort to the adjoint model approach [24] that provides a systematic framework for error backpropagation through arbitrary network structures. This is achieved by unfolding the network in time, which allows apportioning the errors among the weights at different time stages in the past. In contrast to RTRL learning, the adjoint approach is advantageous since it employs a unique model for calculating the error gradients with respect to all the adjustable parameters. The weights update is completed in a single backward run, thus reducing considerably the computational and storage demands. Nevertheless, the gradient descent methods exhibit long convergence times due to the small learning rates required, and are often become trapped to local minima of the error function. In order to overcome the above shortcomings and to cope with the temporal complexities of our application, we employ an on-line decoupled recursive prediction error (DRPE) algorithm for training of the LF-DFNN models. The DRPE scheme is derived by dividing the weight set into several weight groups. A group is used for the premise parameters while the remaining ones are used for grouping of the adjustable weights of the rule sub-models, decomposed at the neuron level. Owing to the second-order information of the error covariance matrices, improved learning qualities are attained with regard to the speed of convergence and solution accuracy.

A real-world application is tackled in this paper, the wind speed forecasting on a site located at the Gulf of Thessaloniki, Northern Greece. The objective is to determine multi-step wind predictions at the park site for up to 3 h ahead, using wind data measured at neighboring sites up to 30 km away (spatial correlation). The availability of efficient wind forecasts allows designing the connection or disconnection of wind turbines or conventional generators, thus attaining low spinning reserve and optimal operational costs. In the past, considerable efforts have been mainly focused on analyzing the wind speed time series of the site under investigation [8]. Nevertheless, the typical statistical properties of wind speed, such as non-stationarity, the gradually decreasing autocorrelation curve and the weak diurnal variation, are not helpful enough. Therefore, research activity was directed to spatial correlation studies, not always leading to satisfactory models. In [5], the spatial correlation of wind turbulence is considered for distances of 700 m–15 km and for time scales of 4, 10 and 30 min. It is concluded that the correlation coefficients strongly depend on the direction of the wind, terrain roughness and height above the ground. For distances of 20–100 km a significant correlation of the hourly or daily average wind speeds has been recognized. This correlation decreases with distance [6] and with the topographical elevation difference [3]. Furthermore, a decrease of the correlation factor is noticed when the wind direction axis differs considerably from the distance vector connecting the measurement sites [23].

The wind forecast models suggested previously are static. To cope with the severe dynamics of the problem, we use the LF-DFNN networks as advanced forecast models producing multi-step wind speed estimates. The spatial and temporal locality of the forecast model along with the richness of the network's dynamics, make it suitable for the identification of the temporal dependencies underlying the process. Additionally, to improve the prediction performance, we employ the DRPE algorithm for model training. The experimental setup includes comparisons of the LF-DFNN forecast models with other recurrent and static neuro-fuzzy network types, and other prediction models used on the same application, previously reported in the literature. To demonstrate the effectiveness of DRPE, comparisons are performed with three conventional gradient descent methods. Initially we proceed to developing the so-called truncated BPTT (T-BPTT), adapted to the LF-DFNN models suggested here. Derivation of the algorithm requires the generation of the corresponding adjoint model, used in the error backpropagation and the weight updates. The obtained adjoint model is also employed with slight modifications for the computation of the ordered partial derivatives of the model's output with respect to its weights, required by DRPE.

The rest of the paper is organized as follows. In Section 2, the architecture of the LF-DFNN model is presented along with the respective notation. In Section 3, derivation of the T-BPTT algorithm is given, including the adjoint model building and parameter update formulas. Section 4 details on the development of the DRPE learning algorithm. In Section 5, simulation results are given and comparative analysis is carried out for the wind-speed forecasting problem. The paper concludes by summarizing the important features of our approach in Section 6.