A Radial Basis Function network training algorithm using a non-symmetric partition of the input space – Application to a Model Predictive Control configuration

Abstract

This work presents the non-symmetric fuzzy means algorithm which is a new methodology for training Radial Basis Function neural network models. The method is based on a non-symmetric fuzzy partition of the space of input variables which results to networks with smaller structures and better approximation capabilities compared to other state-of-the-art training procedures. The lower modeling error and the smaller size of the produced models become particularly important when they are used in online applications. This is demonstrated by integrating the model produced by the proposed algorithm in a Model Predictive Control configuration, resulting in better control performance and shorter computational times.

Introduction

Radial Basis Function (RBF) neural networks constitute a special neural network architecture that has received much attention from the academic community. A typical RBF network comprises only one hidden layer of neurons which results to fewer synaptic weights and in general a less complex structure compared to other neural network architectures. Typically there are two approaches for training an RBF network: The first approach aims at determining all the network parameters in one step using a nonlinear optimization procedure. Following this approach, Sarimveis et al. [19] developed a special genetic algorithm to auto-configure the structure of the network and obtain the model parameters. Hefny et al. [6] introduced a fuzzy neural network which can be considered as a logical version of RBF networks, where genetic algorithms are employed as the learning mechanism. Peng et al. [17] proposed a hybrid forward algorithm for the construction of RBF neural networks with tunable nodes, which according to the authors leads to an improved network performance and reduced memory usage for the network construction. Wedge et al. [22] presented a hybrid RBF-sigmoid neural network with a three-step training algorithm that utilizes both global search and gradient descent training. Their algorithm is intended to identify global features of an input–output relationship before adding local detail to the approximating function. As an alternative to these gradient-based procedures, but still calculating the network parameters in one step, Lazaro et al. [10] used the Expectation–Maximization algorithm to obtain maximum likelihood estimates of the RBF network parameters.

The second approach for training an RBF network is to separate the problem of identifying the network parameters in two steps: The first step aims at finding the number and locations of the hidden node RBF centers, while in the second step the synaptic weights are determined. As the second step is performed trivially by linear regression between the outputs of the hidden node and the real output data, this two-step procedure is usually faster than optimizing all the RBF network parameters at the same time.

Still, determining the number and the locations of the hidden node centers is not an easy task and numerous attempts to solve this problem have been presented in the literature. Among the most recent ones, Panchapakesan et al. [14] proposed a methodology where the locations of the centers are selected using unsupervised methods and then are fined-tuned to decrease the training error. Jiang et al. [8] used a genetic algorithm to select the appropriate network structure in determining the optimal number of nodes. Zhao and Huang [20] introduced an evolutionary structure optimization method for selecting the RBF hidden centers and widths. Sarimveis et al. [18] proposed the fuzzy means algorithm which determines the RBF centers of the network based on a fuzzy partition of the input space; later an adaptive version of the algorithm was proposed for modeling time-varying systems [1]. The main advantage of the algorithm is that it has the ability to determine both the centers and structure of the network in very short computational times, while comparisons with other training methodologies show that the prediction capabilities of the produced models are similar or superior.

The fuzzy means algorithm selects the hidden node centers by partitioning the input space to an equal number of fuzzy sets for each input variable. However, it might be possible to improve the results in terms of prediction accuracy and/or network size, by assigning a different number of fuzzy sets to each input variable. The modification of the original method in order to take into account non-symmetric fuzzy partitions of the input space is not trivial. The goal of this paper is to investigate this possibility by using hyper-ellipsoid fuzzy subspaces instead of the hyper-spherical shapes on which the original algorithm was based. Also the concept of the relative Euclidean distance introduced by Nie [13], is extended to account for the non-symmetric partition.

The main result of this study is the development of the non-symmetric fuzzy means algorithm which is a new training. The algorithm improves the prediction accuracy of the produced models, while at the same time a significant reduction of the number of hidden nodes is achieved. Reduced complexity is a desired model feature, especially when the model is used for real time applications. In order to demonstrate the benefits of the non-symmetric algorithm, the proposed method is combined with a standard Model Predictive Control (MPC) configuration. MPC algorithms make use of a model of the controlled process in order to estimate the optimum sequence of control moves that will drive the process closer to the set-point [7]. Incorporating RBF dynamical models in such a control configuration is not a new concept. Peng et al. [15], [16] presented a methodology for training RBF-based AutoRegressive models with eXogenuous (ARX) variables and then implemented it in the design of an MPC for nonlinear industrial processes. In Alexandridis and Sarimveis [2], an adaptive nonlinear control scheme was introduced, by integrating the symmetric adaptive fuzzy means algorithm into the MPC framework. The complete control scheme was then applied successfully to a digester reactor. The impact of the network size on the computational time needed for solving the online optimization problem was not investigated in details, since for the particular application there was sufficient time between taking two consecutive control actions. However in applications with fast dynamics, the computational time for solving the optimization problem is critical. In such cases it is necessary to find ways to speed up the optimization task. Bhartiya and Whiteley [3], [4] proposed an MPC approach, where a factorized RBF network serves as the process model. In their work the factorization of the RBF network increases the computational efficiency of the control algorithm thus helping to solve the optimization problem faster. Wang et al. [21] applied an MPC methodology based on RBF networks for the adaptive control of the air-fuel ratio in a spark ignition engine. Using a reduced Hessian method, they managed to speed up the nonlinear optimization problem. In this paper we show that utilization of the non-symmetric fuzzy means algorithm results to a model with better accuracy, which guarantees a better control performance, but also decreases the computational burden for solving the on-line MPC optimization problem. This result is verified by applying the proposed MPC configuration to a Continuous Stirred Tank Reactor (CSTR) model.

The rest of this article is organized as follows: A short introduction to the general concept of the fuzzy means algorithm is given in the next section, followed by the presentation of the proposed non-symmetric modification. Next follows a description of how RBF models can be integrated within the MPC configuration and an analysis of the computational complexity of the resulting optimization problem. Then, two case studies are presented, where the proposed methodology is applied to benchmark modeling and control problems and compared with the original version of the fuzzy means algorithm. In the final section we draw conclusions and set some directions for future research.