A soft computing method to predict sludge volume index based on a recurrent self-organizing neural network
Graphical abstract
Introduction
Sludge bulking, due to excessive growth of filamentous bacteria, is one of the most common solid separation problem in the wastewater treatment process (WWTP) [1], [2]. The term bulking is defined as the phenomenon in which the density of activated sludge tends to decrease as a consequence of the overabundance of filamentous bacteria [3]. Despite the fact that sludge bulking has a significant impact on activated sludge system performance, the kinetic properties of filamentous bacteria cannot be considered satisfactory, which makes a mechanistic description of sludge bulking in a general model difficult [4].
To circumvent the lack of a mechanistic understanding of WWTP, the soft computing methods are becoming more common for predicting the water qualities in WWTP, even though they are still not as widespread as, for instance, in the process industry, where the soft computing methods are extensively exploited [5], [6]. In the soft computing methods, the input information and the internal model are used to return output information associated with the hard-to-measure primary variables [7]. For the soft computing methods, models based on the artificial neural network (ANN) and the fuzzy system are the most popular ones [8], [9]. An ANN contains the nodes arranged in layers and connected to each other. The input–output relationship is encoded in the connection weights, which are adapted to minimize the error between the network outputs and the targets [10], [11]. In general, the structures of ANN can be classified as feedforward neural networks (FNNs) and recurrent neural networks (RNNs) [12], [13]. And most of the publications in the soft computing methods use FNNs with backpropagation or its other variations for modeling nonlinear system. However, one of the main drawbacks of FNNs is that they are essentially static input-to-output maps and their capability for representing nonlinear systems is limited [14]. RNNs, on the other hand, are capable of providing long-range predictions even in the presence of measurement noise and own some other advantages due to their recurrent structures [15]. Therefore, RNNs, which have the capability of capturing various plant nonlinearities are better suitable to predict nonlinear systems for the soft computing methods [16]. However, the number of hidden nodes in RNNs is often assumed to be constant [17]. In fact, if the number of hidden nodes is too large, the computational loading is heavy and the generality is poor; on the other hand, if the number of hidden nodes is too small, the learning performance may not be good enough to achieve the desired performance [18], [19]. For these reasons, it is crucial to optimize the structures of RNNs to improve their performance [20].
Generally, there are three ways for designing the structures of RNNs: growing, pruning, and a combination of growing and pruning. The growing methods start with a small size network and add new hidden nodes to the network in the training procedure. For example, Subrahmanya et al. proposed a constructive method for simultaneous structure and parameters training of RNNs based on the particle swarm optimization and covariance matrix adaptation strategy [21]. An Elman-based self-organizing RBF neural network (ESRNN) is designed through the simultaneous structure and parameter learning methods by the Mahalanobis distance approach, in [22]. Furthermore, some other growing RNNs have already been proposed, in [23], [24], [25] and have been shown to outperform fixed RNNs. However, it is possible that the constructive methods may overestimate the number of hidden nodes required [21]. Moreover, the constructive algorithms tend to build small networks due to their incremental learning nature [26]. On the other side, the pruning algorithms are used to delete unnecessary hidden nodes and connections from the oversized RNNs. The pruning algorithms start with an oversized network and remove unnecessary network parameters, either during training or after convergence to a local minimum [27]. In the literature, many different pruning methods have emerged. Zheng et al. introduce an interplay of spike timing-dependent plasticity and different homeostatic mechanisms for designing a class of recurrent self-organizing networks [28]. Leung et al. proposed a local extended Kalman filtering training approach for pruning RNNs [29]. And a recursive Bayesian Levenberg-Marquardt algorithm, which can evaluate the significance of hidden nodes, is proposed to organize RNNs in [30]. Moreover, some other pruning RNNs have also been proposed in [31], [32]. Although these pruning RNNs have several benefits, such as little redundancy in connections or nodes and few parameters. The computational cost is higher since the majority of the training time is spent on networks larger than necessary [33].
Recently, the combination of growing and pruning has become popular for designing RNNs. The growing–pruning algorithm is a hybrid approach, which executes a constructive phase (or a pruning phase) first, and then a pruning phase (or a constructive phase). Therefore, the growing-pruning algorithms effectively enable address some drawbacks of the independent growing or pruning algorithms. Based on the firing strength and the significance of each hidden node, Hsu et al. proposed a dynamic recurrent fuzzy neural network, which can generate or prune the hidden nodes dynamically online for achieving the optimal neural structure [34]. The results show that the dynamic recurrent fuzzy neural network can improve the performance. El-Sousy developed a recurrent self-evolving fuzzy neural network (RRSEFNN), which can perform the structure and parameter learning concurrently [35]. The simulation and experimental results confirm that the proposed RRSEFNN grants robust performance regardless of load disturbances. However, many constraint conditions should be assumed for these self-organizing RNNs in the learning process. Wang et al. proposed an optimization algorithm for an Elman-type RNN, combining the advantages of discrete particle swarm optimization algorithm and improved particle swarm optimization algorithm [36]. The results show that this self-organizing Elman-type RNN can obtain a low architectural complexity and good generalization performance. However, the convergence of the neural network is not discussed and the training process may stop in the early phases. Moreover, to deal with the dynamics of RNNs by introducing the information process of neural systems at the network and whole brain levels, in recent years, many biological methods have been developed for adjusting the network structures underlying observed information signals and the effective connectivity [37], [38]. However, in general, how to design an effective method for RNNs with the efficient performance is still a challenge [39], [40].
With above mentioned motivations in this paper, a growing and pruning RNN called recurrent self-organizing neural network (RSONN) is developed for the soft computing method to predict the values of sludge volume index (SVI) in WWTP. The proposed RSONN-based soft computing method contains three major contributions as follows. First, the growing and pruning method is developed by the sensitivity analysis (SA) method, which uses not only the past states but also the current states for computing the contribution of hidden nodes (previous studies provided strong evidence that compatibly used past and current states to be more desirable for optimizing the structures of RNNs [41], [42]). Then, the structure of RSONN can be self-organized based on the contributions of each hidden node. Secondly, to guarantee the successful applications of the proposed soft computing method, the appropriately adjust learning rates of RSONN is derived based on the Lyapunov stability theorem. Moreover, the convergence of the proposed RSONN is discussed in this paper. Thirdly, an experimental hardware, including the proposed soft computing method is set up. A real WWTP is employed to test the proposed RSONN-based soft computing method. The experimental results have confirmed that the soft computing method exhibits satisfactory predicting performance for SVI.
The rest of this paper is organized as follows: An overview of RNN is given in Section II. The growing and pruning method as well as the parameter learning algorithm of RSONN are described in Section III. The convergence of RSONN is discussed in Section IV. In addition, the appropriately adjust learning rates are given. In section V, an experimental hardware is set up. And the proposed RSONN-based soft computing method is applied to predicting the SVI values in a real WWTP. Finally, the conclusion is given in section VI.
Section snippets
Recurrent neural network (RNN)
A RNN contains input layer, hidden layer, and output layer, as well as feedback connection weights, activation functions, and interconnection weights. In this study, the proposed RNN is designed by the combination of the locally recurrent and globally feed forward structure. The dynamic properties are achieved by utilizing the internal feedbacks as shown in Fig. 1.
For a clear understanding of the computational model for the proposed RNN through the proposed structure, the mathematical function
Recurrent self-organizing neural network (RSONN)
The basic idea of RSONN is to utilize the SA method for designing a suitable network structure. In fact, the SA method, which is well-known in the engineering literature, is to ascertain how the model output depends on its input factors [43]. Inspired by the advantages of the SA method, the significances of hidden nodes can be obtained by their input parameters. Then, the hidden nodes can be pruned or the new hidden nodes can be added in RNNs.
Convergence analysis
For RSONN, the convergence with the structure and weights adjustment is important and requires careful investigation, as it is crucial for the successful applications. To analyze the convergence of the proposed RSONN, this section first discusses the convergence of RSONN with fixed number of hidden nodes. Then, the convergence of RSONN with different number of hidden nodes is studied.
In this study, the Lyapunov function candidate is defined as:where e(t) = yd(t) − y(t) represents the
Simulation studies
In order to demonstrate the effectiveness of the proposed RSONN-based soft computing method, the experimental hardware, including the online sensors and the predicting plant is schematically designed in a real WWTP for predicting the values of SVI. The performance of this proposed method is evaluated by comparing with other methods. All the simulations are programmed with MATLAB version 2014, and were run on a Pentium 4 with a clock speed of 2.6 GHZ and 1 GB of the RAM, under a Microsoft Windows
Discussion
For comparison purpose, the mathematic method [46], the dynamic ARX method [47], the fixed feedforward neural network [48], the support vector machine method [49], the fixed fuzzy neural network [50] and the fuzzy k-nearest neighbors method [51] are used in case 1. Table 3 shows that the RSONN-based soft computing method can obtain higher mean accuracy than the mathematic method, the dynamic ARX method, the feedforward neural network, the support vector machine method, the fixed fuzzy neural
Conclusion
The approaches to the monitoring problem in WWTP rely upon the online and offline analysis of the primary variables. SVI, an index of the sludge bulking, is hard-to-measure and its availability is often associated with expensive capital and maintenance costs. In order to obtain the online values of SVI, a soft computing method is developed to predict the SVI values in a real WWTP. And an RSONN, whose structure and parameters are adjusted concurrently in the training process is designed to
Acknowledgment
The authors would like to thank Prof. Feng Gang for reading the manuscript and providing valuable comments. The authors also would like to thank the reviewers for their valuable comments and suggestions, which helped improve this paper greatly.
References (52)
- et al.
Research advances and challenges in one-dimensional modeling of secondary settling tanks—a critical review
Water Res.
(2014) - et al.
Rheological characterisation of municipal sludge: a review
Water Res.
(2013) - et al.
Filament content threshold for activated sludge bulking: artifact or reality?
Water Res.
(2007) - et al.
A new settling velocity model to describe secondary sedimentation
Water Res.
(2014) - et al.
Prediction of effluent quality of a paper mill wastewater treatment using an adaptive network-based fuzzy inference system
Appl. Soft Comput.
(2011) - et al.
Data-derived soft-sensors for biological wastewater treatment plants: an overview
Environmental Modelling & Software
(2013) - et al.
Modeling nonlinear elastic behavior of reinforced soil using artificial neural networks
Appl. Soft Comput.
(2009) - et al.
An adaptive neuro-fuzzy identification model for the detection of meat spoilage
Appl. Soft Comput.
(2014) - et al.
An efficient self-organizing RBF neural network for water quality prediction
Neural Net.
(2011) - et al.
Robust feedforward and recurrent neural network based dynamic weighted combination models for software reliability prediction
Appl. Soft Comput.
(2014)
An adaptive recurrent fuzzy system for nonlinear identification
Appl. Soft Comput.
Constructive training of recurrent neural networks using hybrid optimization
Neurocomputing
Adaptive backstepping Elman-based neural control for unknown nonlinear systems
Neurocomputing
A multiobjective genetic algorithm for obtaining the optimal size of a recurrent neural network for grammatical inference
Pattern Recogn.
Structure optimization of bilinear recurrent neural networks and its application to ethernet network traffic prediction
Inform. Sci.
A growing and pruning sequential learning algorithm of hyper basis function neural network for function approximation
Neural Netw.
Pruning and regularization in reservoir computing
Neurocomputing
Adaptive hybrid control system using a recurrent RBFN-based self-evolving fuzzy-neural-network for PMSM servo drives
Appl. Soft Comput.
A hybrid optimization-based recurrent neural network for real-time data prediction
Neurocomputing
Sensitivity analysis practices: Strategies for model-based inference
Reliab. Eng. Syst. Safe.
Parameter uncertainty effects on variance-based sensitivity analysis
Reliab. Eng. Syst. Safe.
Comparison and evaluation of empirical zone settling velocity parameters based on sludge volume index using a unified settling characteristics database
Water Res.
Dynamic modeling of filamentous bulking in lab-scale activated sludge processes
J. Process Contr.
The design methodology of radial basis function neural networks based on fuzzy K-nearest neighbors approach
Fuzzy Set. Syst.
Improving over-fitting in ensemble regression by imprecise probabilities
Inform. Sci.
A sensor-software based on a genetic algorithm-based neural fuzzy system for modelling and simulating a wastewater treatment process
Appl. Soft Comput.
Cited by (54)
Sludge bulking monitoring in industrial wastewater treatment plants through graphical methods: A dynamic graph embedding and Bayesian networks approach
2023, Journal of Environmental ManagementSelf-organizing pipelined recurrent wavelet neural network for time series prediction
2023, Expert Systems with ApplicationsUse of artificial neural network for predicting effluent quality parameters and enabling wastewater reuse for climate change resilience – A case from Jordan
2021, Journal of Water Process EngineeringInput variable selection using machine learning and global sensitivity methods for the control of sludge bulking in a wastewater treatment plant
2021, Computers and Chemical EngineeringA soft measurement approach of wastewater treatment process by lion swarm optimizer-based extreme learning machine
2021, Measurement: Journal of the International Measurement ConfederationCitation Excerpt :With the condition that the use of sensors is limited, to complete more real-time measurement of some process variables, soft measurement approaches are widely used [10–12]. In recent years, many machine learning algorithms have been introduced into the construction of soft measurement approaches, such as adaptive neuro-fuzzy inference system (ANFIS) [13], polynomial neural network (PNN) [14], Gaussian process regression (GPR) [15], recurrent neural network (RNN) [16], T-S fuzzy neural network (TSFNN) [17], relevant vector machine (RVM) [18], random forest [19,20] and extreme learning machine (ELM) [21]. In 2004, Huang et al. [22] proposed extreme learning machine (ELM).
Modeling of a real industrial wastewater treatment plant based on aerated lagoon using a neuro-evolutive technique
2021, Process Safety and Environmental ProtectionCitation Excerpt :Literature presents different approaches to develop models for solving specific problems of aerated lagoons used for wastewater treatment: computational fluid dynamics (Morales-Mora et al., 2015; Pougatch et al., 2007), cubic spline (Sibil et al., 2014), linear regression (Escalas-Cañellas et al., 2008; Oliveira-Esquerre et al., 2004), hydraulic and biokinetic models (Houweling et al., 2008; Magbanua et al., 2003a), empirical model (Houweling et al., 2007), equilibrium and thermal balance (Gillot and Vanrolleghem, 2003), artificial neural networks (Oliveira-Esquerre et al., 2004, 2003). If for WWTPs modeling Artificial Intelligence techniques like Artificial Neural Networks (ANNs) and Fuzzy Systems are extensively and efficiently applied (Zeinolabedini and Najafzadeh, 2019; Najafzadeh and Zeinolabedini, 2019; Khatri et al., 2020) (Han et al., 2016; Zhang and Qiao, 2015; Han et al., 2014a; Han and Qiao, 2014; Han et al., 2014b; Liu et al., 2013; Qiao et al., 2012; Jami et al., 2012; Cong et al., 2009; Pai, 2008; Skowronski et al., 2006), for aerated lagoons modeling, the applications of such methods are scarce. Except their performance and ease of use, this is one of the motivating factors that lead to the selection of an ANN based approach for the current work.