Neural networks and genetic algorithms can support human supervisory control to reduce fossil fuel power plant emissions

Abstract

Artificial neural networks and genetic algorithms are two intelligent approaches initially targeted to model human information processing and natural evolutionary process, with the aim of using the models in problem solving. During the last decade these two intelligent approaches have been widely applied to a variety of social, economic and engineering systems. In this paper, they have been shown as modelling tools to support human supervisory control to reduce fossil fuel power plant emissions, particularly NO_x emissions. Human supervisory control of fossil fuel power generation plants has been studied, and the need of an advisory system for operator support is emphasized. Plant modelling is an important block in such an advisory system and is the key issue of this study. In particular, three artificial neural network models and a genetic algorithm-based grey-box model have been built to model and predict the NO_x emissions in a coal-fired power plant. In non-linear dynamic system modelling, training data is always limited and cannot cover all system dynamics; therefore the generalization performance of the resultant model over unseen data is the focus of this study. These models will then be used in the advisory system to support human operators on aspects such as task analysis, condition monitoring and operation optimization, with the aim of improving thermal efficiency, reducing pollutant emissions and ensuring that the power system runs safely.

Appendix: An ANN configuration selection algorithm

Suppose we have a network model with p input nodes and q output nodes. A data set Z₁ is used for neural network training:

$$ \left\{ {\matrix{ {Z_{\bf 1}^{} = \left\{ {z_{1i} ,{\rm i} = {\rm 1}{\rm ,2},...,{\rm N}_1 } \right\}} \cr {z_{1i} = \{ [u_{11} (i){\rm u}_{12} (i){\rm }...{\rm }u_{1p} (i)]^T ;[t_{11} (i){\rm t}_{12} (i){\rm }...{\rm t}_{1q} (i)]^T \} } \cr } } \right. $$

(22)

where $z_{1i} ,{\rm i} = {\rm 1}{\rm ,2},...,{\rm N}_{\rm 1}$ are data samples; $u_{1i} (j),{\rm i} = {\rm 1}{\rm ,2},...,{\rm p}{\rm , j} = {\rm 1}{\rm ,2},...,{\rm N}_{\rm 1}$ are input values; $t_{1i} (j),{\rm i} = {\rm 1}{\rm ,2},...,{\rm q}{\rm , j} = {\rm 1}{\rm ,2},...,{\rm N}_{\rm 1}$ are the output targets.

Then the cost function is defined as

$$ E_1 ({\bf Z}_1 ;\omega ) = \sum\limits_{j = 1}^{N_1 } {\sum\limits_{i = 1}^q {(y_{1i}^{} (j) - t_{1i}^{} (j))^2 } } = \left\| {\varepsilon _1 } \right\|^2 $$

(23)

where $y_{1i} (j),{\rm i} = {\rm 1}{\rm ,2},...,{\rm q}{\rm , j} = {\rm 1}{\rm ,2},...,{\rm N}_{\rm 1}$ are the outputs of the network model given inputs from the training data set, q is the number of output nodes, ω is the adjustable weight vector, e₁ are error vectors.

A recursive training algorithm to update the weights with respect to the cost function defined in (23) may take the following form:

$$ \left\{ \matrix{ \omega ^{(i + 1)} = \omega ^{(i)} - \mu ^{(i)} {\bf{H}}^{(i)} {\rm{E'}}_{\rm{1}} (\omega ^{(i)} ),{\rm{ j}} = 0,{\rm{1}}{\rm{,2}} \hfill \cr \omega ^{(0)} = \omega _0 \hfill \cr} \right. $$

(24)

where µ is the step size, which is determined by some search along the indicated line. H is some positive definite matrix, $E'_1$ is the first derivative of the cost function of (23) with respect to the weights. The initial value for ω could be some prior guess, i.e. ω₀ is generated randomly.

Most recursive learning algorithms are based on Newton-type gradient-descent type techniques. This is a simple gradient descent method but it suffers from the problems of slow convergence and is subject to frequent failure. Consequently, many researchers have developed heuristical extensions such as random search methods. Most recent advances in training have used powerful second-order optimization techniques, and typically involve the calculation of at least an approximate Hessian matrix associated with the function to be optimized. One of these powerful and successful algorithms is the LM (Levenberg–Marquart) method, which is able to produce quick convergence to the optimal solution. The LM method will be used in network training for all three types of neural network models, which will be discussed in the following.

For the LM approach (Hagen and Menhaj 1994), (24) will take the form of

$$ \omega ^{(i + 1)} = \omega ^{(i)} - \mu ^{(i)} ({\rm P}_j^T {\bf P}_j^T + \lambda {\bf I})^{ - 1} {\bf P}_j^T \varepsilon _1 (\omega ^{(i)} ) $$

(25)

where ${\bf P}_j = ({{\partial \varepsilon _j } \over {\partial \omega }})$, I is an identity matrix, ε_l is defined in (23) and λ is a small positive real number.

Now consider that we have two ANN architectures denoted as $ANN( \bullet ;\omega _1 )$ and $ANN( \bullet ;\omega _2 )$, where $\omega _1 \in {\bf R}^{n_1 }$ and $\omega _2 \in {\bf R}^{n_2 }$ are adjustable vectors with different dimensions corresponding to different architecture selection of ANN. The performances of these two trained ANNs are denoted as $E(\omega _1^ * )$ and $E(\omega _2^ * )$ respectively. Then we can define a likelihood-ratio test statistic L:

$$ L = {{\left( {{{E(\omega _2^ * ) - E(\omega _1^ * )} \over {n_1 - n_2 }}} \right)} \mathord{\left/ {\vphantom {{\left( {{{E(\omega _2^ * ) - E(\omega _1^ * )} \over {n_1 - n_2 }}} \right)} {\left( {{{E(\omega _1^ * )} \over {s - n_1 }}} \right)}}} \right. \kern-\nulldelimiterspace} {\left( {{{E(\omega _1^ * )} \over {s - n_1 }}} \right)}} $$

(26)

where s is the number of total samples, and n ₁ and n ₂ are dimensions of adjustable vectors and ω₂ respectively. Although likelihood-ratio test statistic L may not be F-distributed, since neural networks may not have a Gaussian error distribution, it is assumed here that likelihood-ratio test statistic L is approximately F-distributed (with DOF=(n ₁ –n ₂ )/(s–n ₁)). Thus, under the F-distribution assumption, if L exceeds α×100 critical point of the F-distribution, ANN architecture selection $ANN( \bullet ;\omega _2 )$ will be rejected. Now likelihood-ratio test statistic L defined in (26) could be used to compare and select between two network architecture selections. The ANN structure selection algorithm is briefed as follows and illustrated in Fig. 20.

Fig. 20.

Flow chart for ANN structure selection algorithm

Algorithm: architecture selection based on likelihood test

• 1. Determine the maximal numbers of hidden nodes that you wish to apply to the ANN architecture.

• 2. Select an ANN architecture with the maximal number of nodes which is large enough, and denoted as $ANN( \bullet ;\omega _1 )$, and let the number of nodes be denoted as N ₁.

• 3. Find the optimal adjustable vector $\omega _1^ *$ of $ANN( \bullet ;\omega _1 )$ obtained by the training algorithm (25) or, in this paper, the particular LM method.

• 4. Calculate $E(\omega _1^ * )$ for $ANN( \bullet ;\omega _1 )$.

• 5. Select another ANN architecture $ANN( \bullet ;\omega _2 )$ with fewer number of nodes than $ANN( \bullet ;\omega _1 )$, let the number of nodes of $ANN( \bullet ;\omega _2 )$ denoted as N ₂. If the number of nodes N ₂ is less than the minimal number, stop and go to the end.

• 6. Find the optimal adjustable vector $\omega _2^ *$ of $ANN( \bullet ;\omega _2 )$.

• 7. Calculate $E(\omega _2^ * )$ for $ANN( \bullet ;\omega _2 )$.

• 8. Test the null hypothesis that the reduced model $ANN( \bullet ;\omega _2 )$ is equivalent to the $ANN( \bullet ;\omega _1 )$ by calculating the likelihood-ratio test statistic L defined in (26).

• 9. If $L \le F_\alpha$, accept the reduced model $ANN( \bullet ;\omega _2 )$ and replace $ANN( \bullet ;\omega _1 )$ with $ANN( \bullet ;\omega _2 )$, select a new $ANN( \bullet ;\omega _2 )$ whose number of nodes are less than the present N ₂, replace the number N ₂ with that new number and go to step 5.

• 10. If $L > F_\alpha$, keep $ANN( \bullet ;\omega _1 )$, and stop. The final network model is described as $ANN( \bullet ;\omega _1^ * )$ with the optimal adjustable vector $\omega _1^ *$ and $ANN( \bullet ;\omega _1^ * )$ could now be employed for use.

Remark

To determine the maximal number of hidden nodes is an important issue in the proposed algorithm, since it is the starting point to prune the network nodes. The decision is, however, mostly based on experience or trial and error. Experience shows that for most engineering systems the maximal number can be chosen to be between 20 and 40.