Simultaneous monitoring of process mean vector and covariance matrix via penalized likelihood estimation

Abstract

In recent years, some authors have incorporated the penalized likelihood estimation into designing multivariate control charts under the premise that in practice typically only a small set of variables actually contributes to changes in the process. The advantage of the penalized likelihood estimation is that it produces sparse and more focused estimates of the unknown population parameters which, when used in a control chart, can improve the performance of the resulting control chart. Nevertheless, the existing works focus on monitoring changes occurring only in the mean vector or only in the covariance matrix. Stemming from the ideas of the generalized likelihood ratio test and the multivariate exponentially weighted moving covariance, new control charts are proposed for simultaneously monitoring the mean vector and the covariance matrix of a multivariate normal process. The performance of the proposed charts is assessed by both Monte-Carlo simulations and a real example.

Introduction

The statistical process control (SPC) has been a major tool in manufacturing for assignable cause detection and variation reduction (Montgomery, 2005). In a multivariate process, unexpected changes in either the process mean vector or the covariance structure among the variables can lead to an increase in process variability. Therefore, the joint monitoring of both the mean vector and the covariance matrix of a multivariate process becomes very important in ensuring the overall process quality.

In multivariate process monitoring, one of the major challenges comes from the high dimensionality. For a $p$-dimensional process, there are $p$ mean components and $p(p+1)/2$ variance/covariance components that any number of these components may go wrong. Therefore, the number of possible combinations of out-of-control (OC) scenarios is usually high, which makes the conventional general-purpose multivariate control charts ineffective. Similar phenomenon has been observed in many contemporary high-dimensional statistical problems. Recent statistical literature has witnessed a blossom of research on proposing penalized methods to deal with high-dimensional data, under the premise that only a sparse set of variables is relevant. See Hastie et al. (2009) and Bühlmann and Van de Geer (2011) for extensive discussion.

As pointed out by Wang and Jiang (2009) and Li et al. (2013), when a change in a multivariate process occurs, it is typically the case in practice that only a small set of the mean or variance/covariance components has changed. That is, a certain sparsity exists in the shifted mean vector or covariance matrix. As a motivating example, we consider the monitoring of wafer quality in semiconductor manufacturing. Fig. 1 illustrates the shape and thickness distribution of a wafer. In industrial practice, the geometric quality of a wafer is characterized by indicators such as total thickness variation (TTV), total indicator reading (TIR), site TIR (STIR), Bow and Warp. More detailed definitions of these quality variables can be found in Li et al. (2013). Among these variables, TTV, TIR, and STIR are calculated from the thickness distribution, while Bow and Warp are calculated from the convex, concave or uneven shape. The thickness and shape of a wafer are affected by different engineering mechanisms. Therefore, the five quality variables could be classified into two groups. When the manufacturing process changes, it may lead to shifts in one of the groups, thus leading to sparse mean or correlation shifts.

In the setting of Multivariate SPC, when the sparsity assumption about the process shift patterns is reasonable and incorporated into the chart design, one can potentially improve the chart performance by adopting a more focused control charting mechanism. For example, Wang and Jiang (2009) designed a Shewhart-type chart, the variable-selection-based multivariate statistical process control (VS-MSPC) chart, for monitoring the mean vector. The VS-MSPC chart first employs the forward-variable-selection method to select a small set of potentially shifted variables, then calculates a $T^2$-based charting statistic to detect mean changes in the small set of variables. Jiang et al. (2012) incorporated the variable-selection procedure with a MEWMA update equation and proposed the VS-MEWMA chart to further improve the chart performance in detecting small mean shifts. Zou and Qiu (2009) proposed to use the Lasso algorithm, instead of the forward-variable-selection procedure for variable selection.

As for monitoring the covariance matrix, Li et al. (2013) recently took advantage of the sparsity of the usual covariance matrix, and proposed a penalized likelihood ratio (PLR) chart. The PLR chart calculates the penalized likelihood ratio of a group of samples and signals an alarm when the likelihood ratio shifts to an abnormal value. The use of the penalized likelihood ratio has the effect of shrinking some of the components in the covariance matrix to zero, thus reducing the effective dimension of the parameters needed to be monitored. In two independent works, Yeh et al. (2012) and Maboudou-Tchao and Diawara (2013) applied the above idea for the covariance matrix monitoring to the case when only individual observations are available. Yeh et al. (2012) modified the penalty function used by Li et al. (2013) and shrank the sample precision matrix toward the in-control (IC) one, rather than to 0 as more commonly seen in the existing literature. This modification is meaningful in the SPC context. Maboudou-Tchao and Diawara (2013) proposed an effective accumulative method by penalizing the precision matrix per se in a slightly different EWMA propagation. Maboudou-Tchao and Agboto (2013) also studied the monitoring of the covariance matrix when the number of observations is fewer than the number of variables. The authors proposed to use the graphical Lasso algorithm to obtain a sparse estimate of the precision matrix, then used the sparse estimate for shift detection.

Although proven efficient, the variable-selection based or penalized likelihood estimation based charts are designed for detecting either just the mean shift or just changes in the covariance matrix. In this work, we are motivated to develop new penalized likelihood estimation based control charts for simultaneously monitoring the mean vector and the covariance matrix of a multivariate process using individual observations.

Some methods have been developed in the literature for the joint monitoring of the process mean and the variability. Hawkins and Zamba (2005) derived the likelihood ratio test for a change in mean and/or variance for normally distributed data, which formed the basis for a single chart thus developed. The advantages of this method include: (a) the chart does not rely on parameter estimates derived from the Phase I observations since the errors in Phase I estimation may lead to uncertain run length distribution of the chart; (b) the chart is in simple form since it uses only one instead of two charts to monitor both the mean and the variance; and (c) the chart can start quickly given a very small number (typically three) of Phase I observations. Therefore, the chart can be easily used to monitor short-run processes with no or limited historical data. Chen et al. (2001) proposed to monitor the mean and the variance of a univariate process using a single EWMA chart. Two EWMA statistics are first designed for standardized mean and transformed variance terms. Since the two EWMA statistics are independent and follow the same standardized normal distribution, the author suggested to monitor the maximum of these two statistics and the resulting control chart triggers an alarm when the maximum is larger than the control limit. This chart is named the MaxEWMA chart. Li et al. (2010) proposed a self-starting chart for the simultaneous monitoring of the process mean and the variance based on the likelihood-ratio statistic and the EWMA procedure. Nevertheless, all of the afore-mentioned charts are designed for univariate processes.

Zhou et al. (2010) proposed to use the generalized likelihood ratio test (GLRT) to monitor patterned mean and variance (non-constant time-varying) changes. In this chart, a likelihood ratio test statistic was derived based on the process mean, which is used for monitoring process mean shifts with unknown patterns. Another likelihood ratio test statistic was also derived for detecting variance changes. Finally, these two likelihood ratio test statistics are put together to form a new vector, which is then monitored to detect changes in both the mean and the variance. However, in this method, only changes in the diagonal variance components can be detected. In addition, in the GLRT test, the shift pattern information in the alternative hypothesis must be completely known.

In this work, we are motivated to develop a novel charting statistic for the joint monitoring of the process mean vector and the covariance matrix of a multivariate process. It is assumed that only individual observations are available at each sampling period. That is, the subgroup size is equal to one. The charting statistic first tries to estimate the process mean vector and the covariance matrix using a penalized likelihood estimation method. The charting statistic is then derived based on a likelihood ratio test. The sparse estimates obtained in this procedure are also helpful to process diagnosis. It is worthwhile to note that in this paper, we modified the penalty for the covariance matrix to be the Frobenius norm of the difference between the estimated covariance matrix and the in-control covariance matrix. While most earlier works concentrated on the precision matrix partly for the sake of the ease of computation, we think the covariance matrix per se is more relevant for industrial practice and gives more informative clues for diagnostics. On the computational side, we modified the algorithm recently proposed by Bien and Tibshirani (2011) by using a shifted version of the soft-thresholding.

The rest of this paper is organized as follows. Section  2 first presents the likelihood ratio and the existing charts for the joint monitoring of the mean vector and the covariance matrix. Two new control charts are then developed based on the penalized likelihood estimates of the process mean vector and the covariance matrix. Adaptive versions of the proposed charts are proposed to circumvent the need of tuning parameter selection. In Section  3, the performance of the proposed charts are studied and compared with the existing charts. Some guidelines for designing the proposed charts are also discussed. Finally, Section  4 concludes this work with suggestions for future research.