Elsevier

Applied Soft Computing

Volume 38, January 2016, Pages 477-486
Applied Soft Computing

A soft computing method to predict sludge volume index based on a recurrent self-organizing neural network

https://doi.org/10.1016/j.asoc.2015.09.051Get rights and content

Highlights

  • The structure of RSONN can be self-organized based on the contributions of each hidden node, which uses not only the past states but also the current states.

  • The appropriately adjusted learning rates of RSONN is derived based on the Lyapunov stability theorem. Moreover, the convergence of the proposed RSONN is discussed.

  • An experimental hardware, including the proposed soft computing method is set up. The experimental results have confirmed that the soft computing method exhibits satisfactory predicting performance for SVI.

Abstract

In this paper, a soft computing method, based on a recurrent self-organizing neural network (RSONN) is proposed for predicting the sludge volume index (SVI) in the wastewater treatment process (WWTP). For this soft computing method, a growing and pruning method is developed to tune the structure of RSONN by the sensitivity analysis (SA) of hidden nodes. The redundant hidden nodes will be removed and the new hidden nodes will be inserted when the SA values of hidden nodes meet the criteria. Then, the structure of RSONN is able to be self-organized to maintain the prediction accuracy. Moreover, the convergence of RSONN is discussed in both the self-organizing phase and the phase following the modification of the structure for the soft computing method. Finally, the proposed soft computing method has been tested and compared to other algorithms by applying it to the problem of predicting SVI in WWTP. Experimental results demonstrate its effectiveness of achieving considerably better predicting performance for SVI values.

Introduction

Sludge bulking, due to excessive growth of filamentous bacteria, is one of the most common solid separation problem in the wastewater treatment process (WWTP) [1], [2]. The term bulking is defined as the phenomenon in which the density of activated sludge tends to decrease as a consequence of the overabundance of filamentous bacteria [3]. Despite the fact that sludge bulking has a significant impact on activated sludge system performance, the kinetic properties of filamentous bacteria cannot be considered satisfactory, which makes a mechanistic description of sludge bulking in a general model difficult [4].

To circumvent the lack of a mechanistic understanding of WWTP, the soft computing methods are becoming more common for predicting the water qualities in WWTP, even though they are still not as widespread as, for instance, in the process industry, where the soft computing methods are extensively exploited [5], [6]. In the soft computing methods, the input information and the internal model are used to return output information associated with the hard-to-measure primary variables [7]. For the soft computing methods, models based on the artificial neural network (ANN) and the fuzzy system are the most popular ones [8], [9]. An ANN contains the nodes arranged in layers and connected to each other. The input–output relationship is encoded in the connection weights, which are adapted to minimize the error between the network outputs and the targets [10], [11]. In general, the structures of ANN can be classified as feedforward neural networks (FNNs) and recurrent neural networks (RNNs) [12], [13]. And most of the publications in the soft computing methods use FNNs with backpropagation or its other variations for modeling nonlinear system. However, one of the main drawbacks of FNNs is that they are essentially static input-to-output maps and their capability for representing nonlinear systems is limited [14]. RNNs, on the other hand, are capable of providing long-range predictions even in the presence of measurement noise and own some other advantages due to their recurrent structures [15]. Therefore, RNNs, which have the capability of capturing various plant nonlinearities are better suitable to predict nonlinear systems for the soft computing methods [16]. However, the number of hidden nodes in RNNs is often assumed to be constant [17]. In fact, if the number of hidden nodes is too large, the computational loading is heavy and the generality is poor; on the other hand, if the number of hidden nodes is too small, the learning performance may not be good enough to achieve the desired performance [18], [19]. For these reasons, it is crucial to optimize the structures of RNNs to improve their performance [20].

Generally, there are three ways for designing the structures of RNNs: growing, pruning, and a combination of growing and pruning. The growing methods start with a small size network and add new hidden nodes to the network in the training procedure. For example, Subrahmanya et al. proposed a constructive method for simultaneous structure and parameters training of RNNs based on the particle swarm optimization and covariance matrix adaptation strategy [21]. An Elman-based self-organizing RBF neural network (ESRNN) is designed through the simultaneous structure and parameter learning methods by the Mahalanobis distance approach, in [22]. Furthermore, some other growing RNNs have already been proposed, in [23], [24], [25] and have been shown to outperform fixed RNNs. However, it is possible that the constructive methods may overestimate the number of hidden nodes required [21]. Moreover, the constructive algorithms tend to build small networks due to their incremental learning nature [26]. On the other side, the pruning algorithms are used to delete unnecessary hidden nodes and connections from the oversized RNNs. The pruning algorithms start with an oversized network and remove unnecessary network parameters, either during training or after convergence to a local minimum [27]. In the literature, many different pruning methods have emerged. Zheng et al. introduce an interplay of spike timing-dependent plasticity and different homeostatic mechanisms for designing a class of recurrent self-organizing networks [28]. Leung et al. proposed a local extended Kalman filtering training approach for pruning RNNs [29]. And a recursive Bayesian Levenberg-Marquardt algorithm, which can evaluate the significance of hidden nodes, is proposed to organize RNNs in [30]. Moreover, some other pruning RNNs have also been proposed in [31], [32]. Although these pruning RNNs have several benefits, such as little redundancy in connections or nodes and few parameters. The computational cost is higher since the majority of the training time is spent on networks larger than necessary [33].

Recently, the combination of growing and pruning has become popular for designing RNNs. The growing–pruning algorithm is a hybrid approach, which executes a constructive phase (or a pruning phase) first, and then a pruning phase (or a constructive phase). Therefore, the growing-pruning algorithms effectively enable address some drawbacks of the independent growing or pruning algorithms. Based on the firing strength and the significance of each hidden node, Hsu et al. proposed a dynamic recurrent fuzzy neural network, which can generate or prune the hidden nodes dynamically online for achieving the optimal neural structure [34]. The results show that the dynamic recurrent fuzzy neural network can improve the performance. El-Sousy developed a recurrent self-evolving fuzzy neural network (RRSEFNN), which can perform the structure and parameter learning concurrently [35]. The simulation and experimental results confirm that the proposed RRSEFNN grants robust performance regardless of load disturbances. However, many constraint conditions should be assumed for these self-organizing RNNs in the learning process. Wang et al. proposed an optimization algorithm for an Elman-type RNN, combining the advantages of discrete particle swarm optimization algorithm and improved particle swarm optimization algorithm [36]. The results show that this self-organizing Elman-type RNN can obtain a low architectural complexity and good generalization performance. However, the convergence of the neural network is not discussed and the training process may stop in the early phases. Moreover, to deal with the dynamics of RNNs by introducing the information process of neural systems at the network and whole brain levels, in recent years, many biological methods have been developed for adjusting the network structures underlying observed information signals and the effective connectivity [37], [38]. However, in general, how to design an effective method for RNNs with the efficient performance is still a challenge [39], [40].

With above mentioned motivations in this paper, a growing and pruning RNN called recurrent self-organizing neural network (RSONN) is developed for the soft computing method to predict the values of sludge volume index (SVI) in WWTP. The proposed RSONN-based soft computing method contains three major contributions as follows. First, the growing and pruning method is developed by the sensitivity analysis (SA) method, which uses not only the past states but also the current states for computing the contribution of hidden nodes (previous studies provided strong evidence that compatibly used past and current states to be more desirable for optimizing the structures of RNNs [41], [42]). Then, the structure of RSONN can be self-organized based on the contributions of each hidden node. Secondly, to guarantee the successful applications of the proposed soft computing method, the appropriately adjust learning rates of RSONN is derived based on the Lyapunov stability theorem. Moreover, the convergence of the proposed RSONN is discussed in this paper. Thirdly, an experimental hardware, including the proposed soft computing method is set up. A real WWTP is employed to test the proposed RSONN-based soft computing method. The experimental results have confirmed that the soft computing method exhibits satisfactory predicting performance for SVI.

The rest of this paper is organized as follows: An overview of RNN is given in Section II. The growing and pruning method as well as the parameter learning algorithm of RSONN are described in Section III. The convergence of RSONN is discussed in Section IV. In addition, the appropriately adjust learning rates are given. In section V, an experimental hardware is set up. And the proposed RSONN-based soft computing method is applied to predicting the SVI values in a real WWTP. Finally, the conclusion is given in section VI.

Section snippets

Recurrent neural network (RNN)

A RNN contains input layer, hidden layer, and output layer, as well as feedback connection weights, activation functions, and interconnection weights. In this study, the proposed RNN is designed by the combination of the locally recurrent and globally feed forward structure. The dynamic properties are achieved by utilizing the internal feedbacks as shown in Fig. 1.

For a clear understanding of the computational model for the proposed RNN through the proposed structure, the mathematical function

Recurrent self-organizing neural network (RSONN)

The basic idea of RSONN is to utilize the SA method for designing a suitable network structure. In fact, the SA method, which is well-known in the engineering literature, is to ascertain how the model output depends on its input factors [43]. Inspired by the advantages of the SA method, the significances of hidden nodes can be obtained by their input parameters. Then, the hidden nodes can be pruned or the new hidden nodes can be added in RNNs.

Convergence analysis

For RSONN, the convergence with the structure and weights adjustment is important and requires careful investigation, as it is crucial for the successful applications. To analyze the convergence of the proposed RSONN, this section first discusses the convergence of RSONN with fixed number of hidden nodes. Then, the convergence of RSONN with different number of hidden nodes is studied.

In this study, the Lyapunov function candidate is defined as:V(t)=12[e(t)]2,where e(t) = yd(t)  y(t) represents the

Simulation studies

In order to demonstrate the effectiveness of the proposed RSONN-based soft computing method, the experimental hardware, including the online sensors and the predicting plant is schematically designed in a real WWTP for predicting the values of SVI. The performance of this proposed method is evaluated by comparing with other methods. All the simulations are programmed with MATLAB version 2014, and were run on a Pentium 4 with a clock speed of 2.6 GHZ and 1 GB of the RAM, under a Microsoft Windows

Discussion

For comparison purpose, the mathematic method [46], the dynamic ARX method [47], the fixed feedforward neural network [48], the support vector machine method [49], the fixed fuzzy neural network [50] and the fuzzy k-nearest neighbors method [51] are used in case 1. Table 3 shows that the RSONN-based soft computing method can obtain higher mean accuracy than the mathematic method, the dynamic ARX method, the feedforward neural network, the support vector machine method, the fixed fuzzy neural

Conclusion

The approaches to the monitoring problem in WWTP rely upon the online and offline analysis of the primary variables. SVI, an index of the sludge bulking, is hard-to-measure and its availability is often associated with expensive capital and maintenance costs. In order to obtain the online values of SVI, a soft computing method is developed to predict the SVI values in a real WWTP. And an RSONN, whose structure and parameters are adjusted concurrently in the training process is designed to

Acknowledgment

The authors would like to thank Prof. Feng Gang for reading the manuscript and providing valuable comments. The authors also would like to thank the reviewers for their valuable comments and suggestions, which helped improve this paper greatly.

References (52)

  • A. Savran

    An adaptive recurrent fuzzy system for nonlinear identification

    Appl. Soft Comput.

    (2007)
  • N. Subrahmanya et al.

    Constructive training of recurrent neural networks using hybrid optimization

    Neurocomputing

    (2010)
  • C.F. Hsu

    Adaptive backstepping Elman-based neural control for unknown nonlinear systems

    Neurocomputing

    (2014)
  • M. Delgado et al.

    A multiobjective genetic algorithm for obtaining the optimal size of a recurrent neural network for grammatical inference

    Pattern Recogn.

    (2005)
  • D.C. Park

    Structure optimization of bilinear recurrent neural networks and its application to ethernet network traffic prediction

    Inform. Sci.

    (2013)
  • N. Vuković et al.

    A growing and pruning sequential learning algorithm of hyper basis function neural network for function approximation

    Neural Netw.

    (2013)
  • X. Dutoit et al.

    Pruning and regularization in reservoir computing

    Neurocomputing

    (2009)
  • F.F.M. El-Sousy

    Adaptive hybrid control system using a recurrent RBFN-based self-evolving fuzzy-neural-network for PMSM servo drives

    Appl. Soft Comput.

    (2014)
  • X.X. Wang et al.

    A hybrid optimization-based recurrent neural network for real-time data prediction

    Neurocomputing

    (2013)
  • A. Saltelli et al.

    Sensitivity analysis practices: Strategies for model-based inference

    Reliab. Eng. Syst. Safe.

    (2006)
  • W. Yu et al.

    Parameter uncertainty effects on variance-based sensitivity analysis

    Reliab. Eng. Syst. Safe.

    (2009)
  • D.L. Giokas et al.

    Comparison and evaluation of empirical zone settling velocity parameters based on sludge volume index using a unified settling characteristics database

    Water Res.

    (2003)
  • I.Y. Smets et al.

    Dynamic modeling of filamentous bulking in lab-scale activated sludge processes

    J. Process Contr.

    (2006)
  • S.B. Roh et al.

    The design methodology of radial basis function neural networks based on fuzzy K-nearest neighbors approach

    Fuzzy Set. Syst.

    (2010)
  • L.V. Utkin et al.

    Improving over-fitting in ensemble regression by imprecise probabilities

    Inform. Sci.

    (2015)
  • M.Z. Huanga et al.

    A sensor-software based on a genetic algorithm-based neural fuzzy system for modelling and simulating a wastewater treatment process

    Appl. Soft Comput.

    (2015)
  • Cited by (54)

    • A soft measurement approach of wastewater treatment process by lion swarm optimizer-based extreme learning machine

      2021, Measurement: Journal of the International Measurement Confederation
      Citation Excerpt :

      With the condition that the use of sensors is limited, to complete more real-time measurement of some process variables, soft measurement approaches are widely used [10–12]. In recent years, many machine learning algorithms have been introduced into the construction of soft measurement approaches, such as adaptive neuro-fuzzy inference system (ANFIS) [13], polynomial neural network (PNN) [14], Gaussian process regression (GPR) [15], recurrent neural network (RNN) [16], T-S fuzzy neural network (TSFNN) [17], relevant vector machine (RVM) [18], random forest [19,20] and extreme learning machine (ELM) [21]. In 2004, Huang et al. [22] proposed extreme learning machine (ELM).

    • Modeling of a real industrial wastewater treatment plant based on aerated lagoon using a neuro-evolutive technique

      2021, Process Safety and Environmental Protection
      Citation Excerpt :

      Literature presents different approaches to develop models for solving specific problems of aerated lagoons used for wastewater treatment: computational fluid dynamics (Morales-Mora et al., 2015; Pougatch et al., 2007), cubic spline (Sibil et al., 2014), linear regression (Escalas-Cañellas et al., 2008; Oliveira-Esquerre et al., 2004), hydraulic and biokinetic models (Houweling et al., 2008; Magbanua et al., 2003a), empirical model (Houweling et al., 2007), equilibrium and thermal balance (Gillot and Vanrolleghem, 2003), artificial neural networks (Oliveira-Esquerre et al., 2004, 2003). If for WWTPs modeling Artificial Intelligence techniques like Artificial Neural Networks (ANNs) and Fuzzy Systems are extensively and efficiently applied (Zeinolabedini and Najafzadeh, 2019; Najafzadeh and Zeinolabedini, 2019; Khatri et al., 2020) (Han et al., 2016; Zhang and Qiao, 2015; Han et al., 2014a; Han and Qiao, 2014; Han et al., 2014b; Liu et al., 2013; Qiao et al., 2012; Jami et al., 2012; Cong et al., 2009; Pai, 2008; Skowronski et al., 2006), for aerated lagoons modeling, the applications of such methods are scarce. Except their performance and ease of use, this is one of the motivating factors that lead to the selection of an ANN based approach for the current work.

    View all citing articles on Scopus
    View full text