Bayesian sequential update for monitoring and control of high-dimensional processes

Abstract

Simultaneous monitoring of multi-dimensional processes becomes much more challenging as the dimension increases, especially when there are only a few or moderate number of process variables that are responsible for the process change, and when the size of change is particularly small. In this paper, we develop an efficient statistical process monitoring methodology in high-dimensional processes based on the Bayesian approach. The key idea of this paper is to sequentially update a posterior distribution of the process parameter of interest through the Bayesian rule. In particular, a sparsity promoting prior distribution of the parameter is applied properly under sparsity, and is sequentially updated in online processing. A Bayesian hierarchical model with a data-driven way of determining the hyperparameters enables the monitoring scheme to be effective to the detection of process shifts and to be efficient to the computational complexity in the high-dimensional processes. Comparison with recently proposed methods for monitoring high-dimensional processes demonstrates the superiority of the proposed method in detecting small shifts. In addition, graphical presentations in tracking the process parameter provide the information about decisions regarding whether a process needs to be adjusted before it triggers alarm.

Appendices

Appendix 1: Proof of Eq. (1)

The joint distribution $p({\mathbf{\tilde{x}}}_{t} ,{\mathbf{\tilde{\mu }}}_{t} )$ can be recursively obtained as

$$ \begin{gathered} p({\mathbf{\tilde{x}}}_{t} ,{\mathbf{\tilde{\mu }}}_{t} ) = p({\mathbf{x}}_{t} |{\mathbf{\tilde{x}}}_{{t - 1}} ,{\mathbf{\tilde{\mu }}}_{t} )p({\mathbf{\tilde{x}}}_{{t - 1}} |{\mathbf{\tilde{\mu }}}_{t} )p({\mathbf{\tilde{\mu }}}_{t} ) \\ = p({\mathbf{x}}_{t} |{\mathbf{\tilde{x}}}_{{t - 1}} ,{\mathbf{\tilde{\mu }}}_{t} )p({\mathbf{\tilde{x}}}_{{t - 1}} |{\mathbf{\tilde{\mu }}}_{{t - 1}} )p({\mathbf{\tilde{\mu }}}_{t} ) \\ = p({\mathbf{x}}_{t} |{\mathbf{\tilde{x}}}_{{t - 1}} ,{\mathbf{\tilde{\mu }}}_{t} )\frac{{p({\mathbf{\tilde{x}}}_{{t - 1}} ,{\mathbf{\tilde{\mu }}}_{{t - 1}} )}}{{p({\mathbf{\tilde{\mu }}}_{{t - 1}} )}}p({\mathbf{\tilde{\mu }}}_{t} ) \\ = p({\mathbf{x}}_{t} |{\mathbf{\tilde{x}}}_{{t - 1}} ,{\mathbf{\tilde{\mu }}}_{t} )\frac{{p({\mathbf{x}}_{{t - 1}} |{\mathbf{\tilde{x}}}_{{t - 2}} ,{\mathbf{\tilde{\mu }}}_{{t - 1}} )\frac{{p({\mathbf{\tilde{x}}}_{{t - 2}} ,{\mathbf{\tilde{\mu }}}_{{t - 2}} )}}{{p({\mathbf{\tilde{\mu }}}_{{t - 2}} )}}p({\mathbf{\tilde{\mu }}}_{{t - 1}} )}}{{p({\mathbf{\tilde{\mu }}}_{{t - 1}} )}}p({\mathbf{\tilde{\mu }}}_{t} ) \\ = p({\mathbf{x}}_{t} |{\mathbf{\tilde{x}}}_{{t - 1}} ,{\mathbf{\tilde{\mu }}}_{t} )p({\mathbf{x}}_{{t - 1}} |{\mathbf{\tilde{x}}}_{{t - 2}} ,{\mathbf{\tilde{\mu }}}_{{t - 1}} )\frac{{p({\mathbf{\tilde{x}}}_{{t - 2}} ,{\mathbf{\tilde{\mu }}}_{{t - 2}} )}}{{p({\mathbf{\tilde{\mu }}}_{{t - 2}} )}}p({\mathbf{\tilde{\mu }}}_{t} ) \\ = \cdots \\ = p({\mathbf{\tilde{\mu }}}_{t} )\prod\limits_{{i = 1}}^{t} {p({\mathbf{x}}_{i} |{\mathbf{\tilde{x}}}_{{i - 1}} ,{\mathbf{\tilde{\mu }}}_{i} )} \\ \end{gathered} $$

(11)

where the density function $p({\mathbf{\tilde{\mu }}}_{t} )$ is

$$ \begin{gathered} p({\mathbf{\tilde{\mu }}}_{t} ) = p({\mathbf{\mu }}_{t} |{\mathbf{\tilde{\mu }}}_{t} )p({\mathbf{\tilde{\mu }}}_{{t - 1}} ) = p({\mathbf{\mu }}_{t} |{\mathbf{\mu }}_{{t - 1}} )p({\mathbf{\tilde{\mu }}}_{{t - 1}} ) \\ = p({\mathbf{\mu }}_{0} )\prod\limits_{{i = 1}}^{t} {p({\mathbf{\mu }}_{i} |{\mathbf{\mu }}_{{i - 1}} )} \\ \end{gathered} $$

In the second equality holds the Markovian property. By plugging it into Eq. (11), the joint probability density, $p({\mathbf{\tilde{x}}}_{t} ,{\mathbf{\tilde{\mu }}}_{t} )$ can be obtained as $p({\mathbf{\tilde{x}}}_{t} ,{\mathbf{\tilde{\mu }}}_{t} ) = p({\mathbf{\mu }}_{0} )\prod\limits_{{i = 1}}^{t} {p({\mathbf{\mu }}_{i} |{\mathbf{\mu }}_{{i - 1}} )} p({\mathbf{x}}_{i} |{\mathbf{\tilde{x}}}_{{i - 1}} ,{\mathbf{\tilde{\mu }}}_{i} ).$

The denominator probability, $p({\mathbf{\tilde{x}}}_{t} ,{\mathbf{\tilde{\mu }}}_{{t - 1}} )$ is written as

$$ \begin{gathered} p({\mathbf{\tilde{x}}}_{t} ,{\mathbf{\tilde{\mu }}}_{{t - 1}} ) = p({\mathbf{\tilde{\mu }}}_{{t - 1}} |{\mathbf{\tilde{x}}}_{t} )p({\mathbf{\tilde{x}}}_{t} ) = p({\mathbf{\tilde{\mu }}}_{{t - 1}} |{\mathbf{\tilde{x}}}_{{t - 1}} )p({\mathbf{\tilde{x}}}_{t} ) \\ = \frac{{p({\mathbf{\tilde{x}}}_{{t - 1}} ,{\mathbf{\tilde{\mu }}}_{{t - 1}} )}}{{p({\mathbf{\tilde{x}}}_{{t - 1}} )}}p({\mathbf{\tilde{x}}}_{t} ) = p({\mathbf{\tilde{x}}}_{{t - 1}} ,{\mathbf{\tilde{\mu }}}_{{t - 1}} )\frac{{p({\mathbf{\tilde{x}}}_{t} )}}{{p({\mathbf{\tilde{x}}}_{{t - 1}} )}} \\ \end{gathered} $$

Then, the posterior is written as

$$ \begin{aligned} p\left( {{\mathbf{\mu }}_{t} |{\mathbf{\tilde{x}}}_{t} ,{\mathbf{\tilde{\mu }}}_{{t - 1}} } \right) = & \frac{{p({\mathbf{\tilde{x}}}_{t} ,{\mathbf{\tilde{\mu }}}_{t} )}}{{p({\mathbf{\tilde{x}}}_{t} ,{\mathbf{\tilde{\mu }}}_{{t - 1}} )}} = \frac{{p({\mathbf{\tilde{x}}}_{t} ,{\mathbf{\tilde{\mu }}}_{t} )}}{{p({\mathbf{\tilde{x}}}_{{t - 1}} ,{\mathbf{\tilde{\mu }}}_{{t - 1}} )}}\frac{{p({\mathbf{\tilde{x}}}_{{t - 1}} )}}{{p({\mathbf{\tilde{x}}}_{t} )}} \\ = & \frac{{p({\mathbf{\tilde{x}}}_{{t - 1}} )}}{{p({\mathbf{\tilde{x}}}_{t} )}}p({\mathbf{\mu }}_{t} |{\mathbf{\mu }}_{{t - 1}} )p({\mathbf{x}}_{t} |{\mathbf{\tilde{x}}}_{{t - 1}} ,{\mathbf{\tilde{\mu }}}_{t} ) \\ \propto & p({\mathbf{\mu }}_{t} |{\mathbf{\mu }}_{{t - 1}} )p({\mathbf{x}}_{t} |{\mathbf{\tilde{x}}}_{{t - 1}} ,{\mathbf{\tilde{\mu }}}_{t} ) \\ \end{aligned} $$

(12)

When the observations, $x_{i}^{'} s$ are all independent and determined only by the current mean, ${\mathbf{\mu }}_{i} ,$ the distribution can be simply written as

$$ \begin{gathered} p\left( {{\mathbf{\mu }}_{t} |{\mathbf{\tilde{x}}}_{t} ,{\mathbf{\tilde{\mu }}}_{{t - 1}} } \right) = \frac{1}{{p({\mathbf{x}}_{t} )}}p({\mathbf{\mu }}_{t} |{\mathbf{\mu }}_{{t - 1}} )p({\mathbf{x}}_{t} |{\mathbf{\mu }}_{t} ) \\ \propto p({\mathbf{\mu }}_{t} |{\mathbf{\mu }}_{{t - 1}} )p({\mathbf{x}}_{t} |{\mathbf{\mu }}_{t} ) \\ \end{gathered} $$

and the proof is done.

Appendix 2: Proof of Eq. (8)

The conditional distribution, $p(\kappa _{{t,i}} |\hat{\mu }_{{t,i}}^{{(n)}} )$ can be obtained through the Bayesian rule as

$$ p(\kappa _{{t,i}} |\hat{\mu }_{{t,i}}^{{(n)}} ) = \frac{{p(\hat{\mu }_{{t,i}}^{{(n)}} |\kappa _{{t,i}} )p(\kappa _{{t,i}} )}}{{p(\hat{\mu }_{{t,i}}^{{(n)}} )}} $$

(13)

where $p(\hat{\mu }_{{t,i}}^{{(n)}} |\kappa _{{t,i}} )$ is Laplacian distribution and $p(\kappa _{{t,i}} )$ is gamma distribution. The probability density, $p(\hat{\mu }_{{t,i}}^{{(n)}} )$ can be obtained by marginalizing in terms of $\kappa _{{t,i}}$ as

$$ \begin{gathered} p(\hat{\mu }_{{t,i}}^{{(n)}} ) = \int_{0}^{\infty } {p(\hat{\mu }_{{t,i}}^{{(n)}} |\kappa _{{t,i}} )p(\kappa _{{t,i}} )d\kappa _{{t,i}} } = \int_{0}^{\infty } {\frac{{\kappa _{{t,i}} }}{2}e^{{ - \kappa _{{t,i}} |\hat{\mu }_{{t,i}}^{{(n)}} - \hat{\mu }_{{t - 1,i}} |}} \cdot \frac{{\beta ^{\alpha } \kappa _{{t,i}}^{{\alpha - 1}} }}{{\Gamma (\alpha )}}e^{{ - \beta \kappa _{{t,i}} }} d\kappa _{{t,i}} } \\ = \frac{1}{2}\int_{0}^{\infty } {e^{{ - \kappa _{{t,i}} (\beta + |\hat{\mu }_{{t,i}}^{{(n)}} - \hat{\mu }_{{t - 1,i}} |)}} \frac{{(\beta + |\hat{\mu }_{{t,i}}^{{(n)}} - \hat{\mu }_{{t - 1,i}} |)^{\alpha } \kappa _{{t,i}}^{{\alpha - 1}} }}{{\Gamma (\alpha )}}\frac{{\beta ^{\alpha } }}{{(\beta + |\hat{\mu }_{{t,i}}^{{(n)}} - \hat{\mu }_{{t - 1,i}} |)^{\alpha } }}} \\ = \frac{1}{2}\frac{{\beta ^{\alpha } }}{{(\beta + |\hat{\mu }_{{t,i}}^{{(n)}} - \hat{\mu }_{{t - 1,i}} |)^{\alpha } }}\frac{\alpha }{{\beta + |\hat{\mu }_{{t,i}}^{{(n)}} - \hat{\mu }_{{t - 1,i}} |}} \\ = \frac{{\alpha \beta ^{\alpha } }}{{2(\beta + |\hat{\mu }_{{t,i}}^{{(n)}} - \hat{\mu }_{{t - 1,i}} |)^{{\alpha + 1}} }} \\ \end{gathered} $$

(14)

By plugging Eq. (12) into (11), we obtain the density function of (11) as a gamma distribution as

$$ p(\kappa _{{t,i}} |\hat{\mu }_{{t,i}}^{{(n)}} ) = \frac{{(\beta + |\hat{\mu }_{{t,i}}^{{(n)}} - \hat{\mu }_{{t - 1,i}} |)^{{\alpha + 1}} \kappa _{{t,i}}^{\alpha } }}{{\Gamma (\alpha + 1)}}e^{{ - (\beta + |\hat{\mu }_{{t,i}}^{{(n)}} - \hat{\mu }_{{t - 1,i}} |)\kappa _{{t,i}} }} $$

Therefore, the expected value can be obtained as

$$ E\left[ {\kappa _{{t,i}} } \right]_{{p(\kappa _{{t,i}} |\hat{\mu }_{{t,i}}^{{(n)}} )}}^{{(n)}} = \frac{{\alpha + 1}}{{\beta + |\mu _{{t,i}}^{{(n)}} - \hat{\mu }_{{t - 1,i}} |}} $$