An efficient self-organizing RBF neural network for water quality prediction

Abstract

This paper presents a flexible structure Radial Basis Function (RBF) neural network (FS-RBFNN) and its application to water quality prediction. The FS-RBFNN can vary its structure dynamically in order to maintain the prediction accuracy. The hidden neurons in the RBF neural network can be added or removed online based on the neuron activity and mutual information (MI), to achieve the appropriate network complexity and maintain overall computational efficiency. The convergence of the algorithm is analyzed in both the dynamic process phase and the phase following the modification of the structure. The proposed FS-RBFNN has been tested and compared to other algorithms by applying it to the problem of identifying a nonlinear dynamic system. Experimental results show that the FS-RBFNN can be used to design an RBF structure which has fewer hidden neurons; the training time is also much faster. The algorithm is applied for predicting water quality in the wastewater treatment process. The results demonstrate its effectiveness.

Introduction

Predicting water quality in the wastewater treatment process can provide a basis for water treatment plant management decisions that can minimize microbial risks and optimize the treatment operation (Ge & Frick, 2009). In many practical situations, however, it is difficult to predict accurately the quality of water in the treatment process due to a lack of knowledge of the parameters used in the process, or the presence of disturbances in the system. Thus, the predictor should take an appropriate action to counteract the presence of disturbances to which the system is subjected and should be able to adjust itself to the changing dynamics of the system. Radial basis function (RBF) neural networks have been successfully applied for solving dynamic system problems, because they can predict the behavior directly from input/output data (Ferrari et al., 2010, Lee and Ko, 2009, Wang and Yu, 2008). However, the number of hidden neurons in these RBF networks (Ferrari et al., 2010, Lee and Ko, 2009, Wang and Yu, 2008) is often assumed to be constant. In fact, if the number of hidden neurons is too large, the computational loading is heavy and, in general, the performance is poor. On the other hand, if the number of hidden neurons is too small, the learning performance may not be good enough to achieve the desired performance. For this reason, it is crucial to optimize the structure of RBF networks to improve performance. A brief review of the existing algorithms is given below.

The resource-allocating network (RAN), proposed by Platt (1991), was the first dynamic structure RBF neural network model. In the training procedure, new Gaussian neurons can be inserted into the hidden layer. Although the growing strategy usually results in a smaller network size than a fixed-size neural network, the RAN may become extremely large because insignificant hidden neurons are not pruned. Yingwei et al. introduced a strategy to prune hidden neurons, whose contribution is relatively small and incorporated this pruning strategy into the RAN, combining it with Extended Kalman Filter (EKF) (Li, Sundararajan, & Saratchandran, 1997). This network is referred to as the minimal resource-allocating network (MRAN); its applications are described in Li, Sundararajan, and Saratchandran (1998). MRAN is a popular tool used to design optimal RBF structures (Panchapakesan, Palaniswami, Ralph, & Manzie, 2002). However, the initial parameters of the new hidden neurons were not considered and the convergence of MRAN was not discussed (Lian, Lee, Sudhoff, & Stanislaw, 2008).

The self-organizing design of RBF neural networks has been discussed by several authors. Huang, Saratchandran, and Sundararajan (2004) proposed a simple sequential learning algorithm for RBF neural networks, which is referred to as the RBF growing and pruning algorithm (GAP-RBF). The original design of GAP-RBF was enhanced to produce a more advanced model known as GGAP-RBF (Huang, Saratchandran, & Sundararajan, 2005). Both GAP-RBF and GGAP-RBF neural networks use a pruning and growing strategy which is based on the “significance” of a neuron and links it to the learning accuracy. The structure of this RBF neural network is simple and the computational time is less than the time used by a conventional RBF. However, when used in practice, GAP-RBF and GGAP-RBF require a complete set of training samples for the training process. In general, it is not possible for designers and technical domain experts to have a priori knowledge of the training samples prior to implementation. Recently, methods based on genetic algorithms (GA) have been used to change the number of hidden neurons (Gonzalez et al., 2003, Wu and Chow, 2007). The great advantage, in theory, of a GA is its ability to do global searching, but this comes at the cost of increased computation requirements. Feng (2006) proposed a self-organizing RBF neural network model based on a particle swarm optimization (PSO) method which attempts to solve the training time issue. The PSO method is used to construct the structure of the RBF neural network in order to simplify and speed up optimization. However, because the PSO method is a population-based evolutionary computation technique, the training time is still too long.

In this paper, a new algorithm, which is called the flexible structure RBF neural network (FS-RBFNN) is presented. This algorithm has several advantages: firstly, a neuron’s average firing rate is used to determine whether new neurons should be inserted. The rate of firing used in the FS-RBFNN is similar to the spiking frequency of the presynaptic neuron in the biological neural system (Neves, Cooke, & Bliss, 2008). When the rate value of the hidden neuron is bigger than a given threshold value, new neurons will be inserted into the hidden layer.

Secondly, the connectivity of hidden neurons is estimated using an information-theoretic methodology. The connectivity between hidden neurons is obtained by measuring the mutual information (MI) (Krivov, Ulanowicz, & Dahiya, 2003) in the training process.

Thirdly, the convergence of the FS-RBFNN is analyzed both theoretically and experimentally. However, most of the self-organizing methods used for RBF structure design (Alexandridis et al., 2003, Bortman and Aladjem, 2009, Shi et al., 2005) only analyze the convergence for learning process by experiments. In order to work in practice it is essential that the FS-RBFNN training process must be convergent. The FS-RBFNN has been specifically designed with this in mind.

Fourthly, the FS-RBFNN is particularly focused on reducing the retraining epochs after the structure has been modified by growing and pruning. The error required (ER) method is used to determine the initial values of the neurons inserted into the FS-RBFNN. It is well known that when a design algorithm adds (or prunes) hidden neurons to (or from) an existing RBF structure, it retrains the modified structure to adapt their connection weights.

The outline of this paper is as follows. Section 2 describes how the MI and the average firing rate are used to design the RBF; it also introduces the FS-RBFNN. Section 3 discusses and analyzes the algorithm. Section 4 presents experimental results which compare the performance of the algorithm with other similar algorithms. Section 5 concludes the paper.