Conditional random field for monitoring multimode processes with stochastic perturbations

https://doi.org/10.1016/j.jfranklin.2020.05.039Get rights and content

Abstract

Multimode process monitoring techniques have been successfully applied in various industrial systems. However, the transition processes between different modes have not been well handled. This paper considers the monitoring problem of multimode processes with stochastic perturbations, in which frequent mode switches caused by stochastic perturbations are taken into account. Contrary to the widely used assumption of instant mode switches, the dynamics during the transition process caused by perturbations are considered. To cope with transient characteristics, a mode identification algorithm based on the conditional random field is proposed. Compared with traditional multimode process monitoring methods, the assumption of independence between adjacent observations is not required, which improves the mode identification accuracy. In addition, an index called log conditional probability ratio is proposed, and its Mahalanobis distance is used for fault detection. The fault detectability of the proposed method is analyzed, with a sufficient condition and a necessary condition of detecting a sensor fault derived, respectively. The effectiveness of mode identification and fault detection is demonstrated by a numerical example and a continuous stirred tank reactor simulation.

Introduction

Data-driven process monitoring techniques have been widely applied in considerable industrial systems with the increasing demand for safety and reliability [1]. Owing to the complexity of modern industrial processes, data-driven methods have attracted a lot of attention during the last several decades because of their advantage that analytical models are not required [2], [3]. This advantage is even more prominent when monitoring large-scale industrial processes. Recently, various data-driven fault diagnosis methods have been proposed to deal with different problems such as adaptive monitoring [4], [5], non-Gaussian process monitoring [6], [7], fault classification [8], [9], incipient fault detection [10] and isolation [11]. Their effectiveness can also be witnessed by recent applications in high-speed trains [12], blast furnace iron-making processes [13], [14], and aluminum electrolysis processes [15], [16].

For most process monitoring problems, there exists a basic assumption that process measurements under normal conditions satisfy a unimodal distribution, for example, Gaussian distribution. However, the operating modes may change from time to time in practical industrial processes, which can be caused by production quota alterations, fluctuations in feedstock or reactant, and external environment changes [17]. Some of the mode changes are known to operators such as manual settings. In this case, the mode information can be directly utilized and the monitoring algorithm can focus on measurements from that specific mode. For some cases where operating modes are unknown, it is necessary to carefully characterize data features from different modes because of the multimodality of data distribution and mode shift characteristics. Generally, there are two types of methods for monitoring multimode processes when mode labels are unknown. One is to directly detect faults without the prerequisite of mode identification. The other is to firstly identify the mode and then use indices from that mode to detect faults.

The majority of the existing multimode process monitoring approaches belong to the first type, in which mode identification is not the prerequisite for fault detection. Since the hard partition of the modes is avoided, some of this type of methods can monitor transition processes. This kind of methods can be further divided into soft modeling methods and adaptive methods. The soft modeling methods combine several local models by probabilistic approaches. Zhao et al. proposed multiple principal component analysis (PCA) models for monitoring multimode processes, in which principal angles are used to measure the similarities between different PCA models [18]. Based on the finite Gaussian mixture model (FGMM) and Bayesian inference strategy, Yu and Qin proposed a Bayesian inference-based probability index to monitor multimode processes [17]. With the assumption that correlations among different modes are similar to each other, Zhao et al. proposed an approach for monitoring processes with between-mode transitions [19]. A mixture Bayesian regularization method based on probabilistic PCA was developed by Ge and Song [20]. Tong et al. presented a multimode process monitoring method using mode clustering and mode unfolding [21]. Tan et al. proposed a nonstationary discrete convolution kernel, to account for the multimodality behavior and improve the fault detection performance [22]. For large-scale processes, Zhang and Zhao developed a distributed and hierarchical monitoring framework, in which the Gaussian mixture model (GMM) is used to handle the varying operation conditions [23]. However, this type of methods is difficult to describe the multimode precisely, since the mode related knowledge is not incorporated in monitoring models. In addition, most of the existing methods ignored the serial correlation between adjacent observations. Another way for multimode process monitoring is to update the model recursively, namely, the adaptive methods. Zhu et al. proposed a recursive mixture factor analyzer for multimode time-variant process modeling and monitoring [24]. Zheng et al. presented a GMM-based adaptive process monitoring method, in which the Bayesian inference probability index is used as the monitoring statistic in both the continuous and batch process monitoring models [25]. However, this kind of methods may accommodate to fault data [26], because it is difficult for the adaptive mechanisms to distinguish mode switches and faults. In addition, since the monitoring models need to be constantly updated online with new observations, the adaptive methods usually require high computational complexity.

For approaches first addressing the problem of mode identification, the identification accuracy is vastly important for subsequent fault detection. From the perspective of machine learning, there are generally two types of methodologies for mode identification, namely, classification and sequence labeling. The main difference lies in whether to treat operating modes as individual samples or a sequence. Both of these two methodologies have been utilized for multimode process monitoring. Shang et al. proposed a method based on conditionally independent Bayesian learning (CIBL) [27] and recursive transformed component statistical analysis (RTCSA) [10] to detect and isolate incipient sensor faults in multimode processes. CIBL is a classification method based on naïve Bayes. It uses an orthogonal transformation to improve the extent of conditional independence, and thus improves the accuracy of mode identification. For nonlinear multimode processes, Wang et al. used a Dirichlet process Gaussian mixed model for mode identification, and a t-distributed stochastic neighbor embedding for process monitoring [28]. However, for many multimode processes, if mode switches depend on previous modes, the mode identification methods based on sequence labeling generally outperform those based on classification because of their difference in coping with mode sequences. As one of the most widely used sequence labeling methods, hidden Markov model (HMM) has many advantages in identifying operating modes in multimode processes, such as the ability to obtain joint probability distribution and the high training efficiency. Yu combined HMM with Mahalanobis distance and negative log-likelihood probability (NLLP) for multimode process monitoring [29]. Ning et al. combined HMM with statistics pattern analysis (SPA) [30] to monitor multimode processes, in which the mode sequence can be determined by the Viterbi algorithm and the monitoring index can switch automatically between NLLP and SPA [31]. For multimode processes with mode-reachability constraints, Afzal et al. used HMM with a two-step Viterbi algorithm to identify the modes [32]. It should be pointed out that one basic assumption of HMM is that the observations should be independent. However, this assumption is seldom satisfied in practical dynamic systems. This contradiction may weaken the ability of HMM in mode identification. The ability for handling transition processes and disadvantages of the existing multimode monitoring methods are summarized in Table 1.

In this paper, we consider the problem of multimode process monitoring, in which the characteristic of multimode is caused by stochastic perturbations. This perturbation can arise from fluctuations of feedstock, reactant, or external environment. For instance, in blast furnace iron-making processes, the quality of iron ores fluctuates frequently, leading to changes of physical and chemical reaction characteristics in the blast furnace. Such fluctuates may trigger mode switches. Compared with traditional multimode processes, the main difference is that mode changes are more frequent due to the stochastic perturbations, and thus the transition processes occur frequently. In such circumstances, the statistical properties of the samples in different modes could be similar, which brings additional difficulties in distinguishing them. To tackle this problem, we propose a multimode process monitoring method based on the conditional random field (CRF). Contrary to HMM, CRF does not require the independence of observations. This advantage makes CRF more suitable for capturing transition processes caused by stochastic perturbations. In addition, CRF has a more flexible formulation with respect to feature functions, which improves its adaptability to systems with different statistical characteristics. For example, the dependence between adjacent measurements can be taken into consideration. Based on CRF, we further design a detection index which is sensitive to faults, completing the procedures of mode identification and fault detection of multimode processes.

To the best of our knowledge, this is the first time that CRF is used in multimode process monitoring, although it has been adopted in the field of fault diagnosis. Recently, Fang et al. proposed a mode diagnosis method based on CRF, which can cope with missing data [33]. The main differences between [33] and this paper are listed in the following. 1) The problems to be solved are different. In [33], it is assumed that only one operating mode is normal, while others are abnormal. When training CRF, both normal and abnormal data are required. In this paper, the multimode problem is considered. All the possible modes are caused by perturbations instead of faults. In other words, all these modes are normal, and the a priori information of faults is not necessarily known. 2) In this paper, a new index is proposed to detect potential faults. That is, CRF is used to label the modes, and fault detection is based on the proposed index. 3) In [33], the observations are discretized using triangular representation, which may lead to information loss when coping with the problem of multimode process monitoring. On the contrary, this paper directly uses original observations.

The main contributions of this paper are summarized as follows. (i) Based on CRF, a new multimode monitoring method is proposed for mode identification and fault detection. The previous modes and observations are involved in the feature functions of the CRF model, such that the prerequisite of independent observations is eliminated. (ii) The detection index is well-designed to ensure its sensitivity to faults. (iii) The fault detectability of the proposed method is analyzed. Specifically, a sufficient condition and a necessary condition for detecting a sensor fault are derived, respectively. (iv) The simulations indicate a higher mode identification accuracy and better detection performance of the proposed method, compared with the state-of-the-art algorithms.

The remainder of this paper is organized as follows. Section 2 formulates the problem. A brief introduction of CRF is given in Section 3. The proposed algorithm is elaborated in Section 4. Section 5 gives the detectability analysis of the proposed approach. In Section 6, simulations are used to verify the performance of the proposed method. Conclusions are given in Section 7.

Section snippets

Problem formulation

Consider a dynamic system equipped with m sensors. In general, a dynamic system refers to a system of elements that change over time. Assume that neither the system model nor a priori process knowledge is available. The system presents multimode characteristics owing to the existence of stochastic perturbations. In a continuous process, each steady state of the process refers to a mode, and the processes between different modes when the mode switches from one to another are called transition

Preliminary

This section gives a brief introduction of CRF. Unlike HMM, CRF is a discriminative model. In fact, a linear-chain CRF and an HMM can be regarded as a discriminative–generative pair. HMM is modeled by joint distribution, i.e., p(Y,C)=p(C)p(Y|C), while CRF is modeled by conditional distribution, namely p(C|Y). Thus, an obvious difference between CRF and HMM is that the distribution of the observations is not involved in CRF modeling. Therefore, no assumptions are required for p(Y), such as

CRF modeling

To extract the dynamic features of the multimode process, both current and previous observations are taken into account. Here, we add an edge between yt1 and ct in the CRF model, as illustrated in Fig. 2. This means that both the current observations and previous observations are considered in mode labeling.

From Fig. 2, there exist three types of edges linking the labels and observations. The first type of edges links ct and ct1. This type of edges is used to describe the relationship between

Detectability

In this subsection, we analyze the detection performance of CRF-LCPR proposed in Section 4. For a sensor fault in the form of constant bias, a sufficient condition and a necessary condition of the detectability are given, respectively. When the process is normal, it is assumed that the detection index seldom exceeds its threshold with a probability of α, which can be regarded as the significance level. Hence, the following detectability analysis is conducted in a probability sense. In other

Numerical simulation

Consider the following dynamic process:{s(k+1)=As(k)+c(k)+v(k)y(k)=Ds(k)+e(k)where s(k)R2 denotes the data source at the kth sampling instant, y(k)R3 is the measurement vector. A and D are coefficient matrices given byA=[0.7170.5010.3140.823]D=[0.3720.6810.4890.2950.9840.179].Assume that the a priori knowledge of both A and D is unavailable. c(k)R2 is a vector which depends on a variable that fluctuates frequently. The variable is simulated by a bounded Brownian motion, which can be

Conclusion

In this paper, a new method CRF-LCPR is proposed for monitoring multimode processes with stochastic perturbations. To address the difficulty in capturing the dynamics of transition processes, a CRF-based mode identification method is provided, in which the dependencies between adjacent observations are taken into account. By leveraging conditional probabilities obtained by CRF, an index called LCPR is proposed and the Mahalanobis distance of LCPRs serves as a fault detection index, which can

Acknowledgment

This work was supported by the National Natural Science Foundation of China (61903326, 61933015), China Postdoctoral Science Foundation (2019M662051), and Zhejiang Postdoctoral Research Foundation (ZJ2019093).

References (38)

Cited by (13)

  • Multimode process monitoring based on hierarchical mode identification and stacked denoising autoencoder

    2022, Chemical Engineering Science
    Citation Excerpt :

    A between-mode relative analysis algorithm (Zhao et al., 2015) decomposed the relative changes between the modes and can effectively distinguish between different types of process changes in different modes for online process monitoring. A multimode process monitoring method based on the conditional random field was proposed(Zhang et al., 2020), which considered the dependencies between adjacent metrics to accomplish mode recognition of multimode process. However, for online applications, the structure of these models is more complex and it is difficult to determine the mode to which the current sample belongs, which can lead to a high level of false alarms if the wrong model is used.

  • Conditional discriminative autoencoder and condition-driven immediate representation of soft transition for monitoring complex nonstationary processes

    2022, Control Engineering Practice
    Citation Excerpt :

    Generally, at the beginning of the switching, characteristics are more similar to the previous steady operation, while at the end, more similar to the next one. Hence, it is unreasonable to strictly divide transition data into one steady operation, which is insufficient to describe the transition characteristics and may lead to false alarms or missing alarms (Dong, Zhang, & Peng, 2021; Tan, Ottewill, & Thornhill, 2019; Tong, Palazoglu, & Yan, 2013; Zhang, Shang, Yang, & Sun, 2020; Zhao et al., 2007). For time-driven methods, some works have been developed to address this problem.

  • A continuous learning monitoring strategy for multi-condition of nuclear power plant

    2021, Annals of Nuclear Energy
    Citation Excerpt :

    This method utilizes variational autoencoder to extract features from input data set containing noise, improves the traditional real-time learning method, and is applied to soft sensor modeling (Guo et al., 2020). Zhang proposed the mode identification algorithm based on the conditional random field (Zhang et al., 2020). Wu proposed a novel self-adaptive deep learning method based on local adaptive standardization and variational auto-encoder bidirectional long short-term memory (LAS-VB).

  • A Dynamic Model-Level Method for Multimode Process Monitoring Using ω-Metric

    2023, IEEE Transactions on Control Systems Technology
View all citing articles on Scopus
View full text