Quality-relevant independent component regression model for virtual sensing application

doi:10.1016/j.compchemeng.2018.04.006

Computers & Chemical Engineering

Volume 115, 12 July 2018, Pages 141-149

https://doi.org/10.1016/j.compchemeng.2018.04.006 Get rights and content

Highlights

•
A new quality-relevant modeling method, QR-ICR, was developed.
•
Two new types of statistical criteria, called cumulative percent relevance (CPR) and Max-Dependency (Max-Dep), were proposed.
•
QR-ICR give more useful information on product quality, thus it is beneficial for quality prediction.
•
By using CPR or Max-Dep, QR-ICR_(CPR) or QR-ICR_(Max-Dep) gives a much simpler model.
•
QR-ICR_(CPR) and QR-ICR_(Max-Dep) achieved better prediction performances than the conventional methods.

Abstract

Independent component regression (ICR) is an efficient method for tackling non-Gaussian problems. In this work, the defects of the conventional ICR are analyzed, and a novel quality-relevant independent component regression (QR-ICR) method based on distance covariance and distance correlation is proposed. QR-ICR extracts independent components (ICs) using a quality-relevant independent component analysis (QR-ICA) algorithm, which simultaneously maximizes the non-Gaussianity of ICs and statistical dependency between ICs and quality variables. Meanwhile, two new types of statistical criteria, called cumulative percent relevance (CPR) and Max-Dependency (Max-Dep), are proposed to rank the order and determine the number of ICs according to their contributions to quality variables. The proposed QR-ICR_(CPR) and QR-ICR_(Max-Dep) methods were validated through a vinyl acetate monomer production process and a benchmark near-infrared spectral data. The results have demonstrated that the proposed QR-ICR_(CPR) and QR-ICR_(Max-Dep) provide simpler predictive models and give better prediction performances than PLS, ICR, ICR_(CPR), and ICR_(Max-Dep).

Introduction

In any industry, it is important to monitor some important quality variables in real time. However, in some cases, online measurement of quality variables is difficult or even impossible due to limitations of hardware analyzers, which are time-consuming maintenance, needing for calibration, aged deterioration, long dead-time, and slow dynamics as clarified via the questionnaire survey in Japan (Kano and Fujiwara, 2013). To solve these problems, virtual sensing technology has been developed and widely used in industrial processes. The basic idea of virtual sensing technology is to estimate the product quality utilizing some easily measured secondary variables. In recent years, data-driven soft-sensors have attracted much more attention because a massive amount of process data are recorded. Among them, multiple linear regression (MLR), principal component regression (PCR), and partial least squares (PLS) are the most popular multivariate statistical modeling approaches (Kadlec, Gabrys, Strandt, 2009, Kano, Ogawa, 2010).

In complex chemical processes, the conventional methods like PCR and PLS are inadequate to describe the underlying data structure because they are second-order methods, which consider only mean and variance or covariance of process data, and cannot cope with non-Gaussianity. To deal with this problem, some non-Gaussian methods like support vector machine (Balabin, Lomakina, 2011, Xu, Lu, Baldea, Edgar, Nixon, 2017), Gaussian mixture model (Yu and Qin, 2008), and independent component analysis (ICA) (Hyvärinen, 1999) have been developed. Among them, ICA is an effective technique for extracting several independent signal components from observed data (Hyvärinen, Oja, 2000, Oja, Nordhausen, 2001). Compared to principal component analysis (PCA), ICA gives a high-order representation for non-Gaussian data. The recovered independent components (ICs) reveal more useful statistical information and make the interpretation of regression model easier as compared to principal components (PCs). ICA has been used for process monitoring, and it is demonstrated that ICA-based methods are superior to PCA-based methods (Kano, Hasebe, Hashimoto, Ohno, 2004, Kano, Tanaka, Hasebe, Hashimoto, Ohno, 2003, Lee, Yoo, Lee, 2004). For quality control, ICA-based regression methods such as independent component regression (ICR) have been proposed. ICR uses the extracted ICs as predictor variables to build a regression model based on the least square criterion. Chen and Wang (2001) used ICR to separate the constituent components from the mixed spectra and estimate their concentrations. Westad (2005) combined ICA and PLS (IC-PLS) for sensory data analysis. Gustafsson (2005) illustrated that ICR yields chemically interpretable latent variables in multivariate regression as compared to PCR and PLS. Shao et al. (2006) have further demonstrated that ICR is a promising tool to retrieve both quantitative and qualitative information from the near-infrared spectra (NIR) data. Kaneko et al. (2008) applied an ICA-MLR method to aqueous solubility data analysis. Zhao et al. (2010) introduced an improved ICR and quantitative calibration algorithm through constructing joint latent variables based regression model. Ge and Song (2014) developed an ensemble ICR model based on different ensemble strategies. Zhang et al. (2015) proposed an ICR-based just-in-time learning model for complex batch process quality prediction. Zheng and Song (2016) proposed a two-level ICR model based on Bayesian inference strategy for multivariate spectroscopic calibration.

The conventional ICR also shows some shortcomings. For example, after extracting ICs from the observed data, there is no standard criterion to decide how many ICs should be retained for building an accurate prediction model. Unlike PCA which ranks the PCs in descending order according to the variance of each PC, ICA cannot decide which ICs are more significant because the extracted ICs have equal variances. In fact, to ensure optimal prediction performance, it is necessary for the engineers to know the magnitude of contribution of each IC to the regression model. In addition, uninformative ICs should not be used in regression models to improve their interpretability, prediction performance, and computational load. Furthermore, for regression analysis, the prediction performance of regression models like ICR actually depends on how well the features (ICs) are extracted from observed data. However, in the conventional ICR algorithm, the extracted ICs take no account of the quality information, thus it may not be optimal for quality prediction.

To deal with these issues, we propose a novel quality-relevant independent component regression (QR-ICR) method based on distance covariance and distance correlation. The basic idea is to first estimate ICs using a quality-relevant independent component analysis (QR-ICA) algorithm which simultaneously maximizes both the non-Gaussianity of ICs and statistical dependency between ICs and quality variables. Second, two new types of statistical criteria, called cumulative percent relevance (CPR) and Max-Dependency (Max-Dep), are designed to rank the order and determine the number of ICs. Compared to the conventional ICR method, QR-ICR can extract ICs which have more useful information on product quality, and thus it is beneficial to improve the prediction performance. In addition, by using CPR or Max-Dep criterion, QR-ICR (here denote as QR-ICR_(CPR) or QR-ICR_(Max-Dep)) gives a much simpler calibration model with fewer significant ICs, and thus the interpretation of the model becomes easier, and the computational load for online prediction becomes lower.

The rest of this paper is organized as follows. Section 2 gives a brief introduction of distance covariance and distance correlation, and ICR. Then, the quality-relevant ICR (QR-ICR) algorithm with two new types of statistical criteria: cumulative percent relevance (CPR) and Max-Dependency (Max-Dep) is presented in Section 3. In Section 4, the proposed QR-ICR_(CPR) and QR-ICR_(Max-Dep) methods are applied to a vinyl acetate monomer production process and a benchmark near-infrared spectral data, and their application results are compared with PLS, ICR, ICR_(CPR), and ICR_(Max-Dep). The conclusions of this work are presented in Section 5.

Section snippets

Distance covariance and distance correlation

Classical Pearson product-moment covariance and correlation (ρ) measure linear dependence between two random variables, and in the bivariate normal case $ρ = 0$ is equivalent to independence. Distance covariance (dcov) and distance correlation (dcor) are new statistics that measure dependence of random vectors, without requiring linear normality assumptions (Székely, Rizzo, Bakirov, et al., 2007, Székely, Rizzo, et al., 2009). The idea of correlation is generalized in the following fundamental

Quality-relevant independent component regression (QR-ICR)

The prediction performance of soft-sensors largely depends on how well the features are extracted from observed data. However, as shown in Fig. 1(a), the extracted features (ICs) in the conventional ICR algorithm take no account of quality information, thus they may not be optimal for quality prediction. In Fig. 1, the “B(W) Learning” block denotes the estimation of the orthogonal matrix B (or the mixing matrix W). In addition, unlike PCA which arranges the PCs in proper order according to

Case studies

In this section, the effectiveness of the proposed QR-ICR_(CPR) and QR-ICR_(Max-Dep) methods was validated through their applications to a vinyl acetate monomer production process and a benchmark NIR spectral data. Here, QR-ICR_(CPR) and QR-ICR_(Max-Dep) denote the proposed QR-ICR using two different types of ICs selection criteria: cumulative percent relevance (CPR) and Max-Dependency (Max-Dep). Moreover, for fair comparison and also for investigating whether the proposed CPR and Max-Dep criteria

Conclusions

In this research, we proposed a novel quality-relevant independent component regression (QR-ICR) method based on distance covariance and distance correlation. Unlike the conventional ICR, QR-ICR extracts ICs with the consideration of quality information, this helps in improving the prediction performance of regression model. In addition, in order to rank the order and determine the number of ICs according to their contribution to quality variables, two types of statistical criteria, called

References (40)

Z. Ge et al.
Ensemble independent component regression models and soft sensing application
Chemom. Intell. Lab. Syst.
(2014)
A. Hyvärinen et al.
Independent component analysis: algorithms and applications
Neural Netw.
(2000)
P. Kadlec et al.
Data-driven soft sensors in the process industry
Comput. Chem. Eng.
(2009)
M. Kano et al.
Evolution of multivariate statistical process control: application of independent component analysis and external analysis
Comput. Chem. Eng.
(2004)
M. Kano et al.
The state of the art in chemical process control in japan: good practice and questionnaire survey
J. Process Control
(2010)
J.-M. Lee et al.
Statistical process monitoring with independent component analysis
J. Process Control
(2004)
M.L. Luyben et al.
An industrial design/control study for the vinyl acetate monomer process
Comput. Chem. Eng.
(1998)
Y. Machida et al.
Vinyl acetate monomer (vam) plant model: a new benchmark problem for control and operation study
IFAC-PapersOnLine
(2016)
S.J. Qin et al.
Determining the number of principal components for best reconstruction
J. Process Control
(2000)
H. Seki et al.
Plantwide control system design of the benchmark vinyl acetate monomer production plant
Comput. Chem. Eng.
(2010)

X. Shao et al.

A new regression method based on independent component analysis

Talanta

(2006)

G.J. Székely et al.

Energy statistics: a class of statistics based on distances

J. Stat. Plan. Inference

(2013)

J. Zheng et al.

Two-level independent component regression model for multivariate spectroscopic calibration

Chemom. Intell. Lab. Syst.

(2016)

R.M. Balabin et al.

Support vector machine regression (svr/ls-svm)-an alternative to neural networks (ann) for analytical chemistry? comparison of nonlinear methods on near infrared (nir) spectroscopy data

Analyst

(2011)

D.P. Bertsekas

Nonlinear Programming

(1999)

J. Chen et al.

A new approach to near-infrared spectral data analysis using independent component analysis

J. Chem. Inf. Comput. Sci.

(2001)

K. Deb

Multi-objective optimization

Search Methodologies

(2014)

J.H. Friedman

Exploratory projection pursuit

J. Am. Stat. Assoc.

(1987)

M.G. Gustafsson

Independent component analysis yields chemically interpretable latent variables in multivariate regression

J. Chem. Inf. Model.

(2005)

A. Hyvärinen

Fast and robust fixed-point algorithms for independent component analysis

IEEE Trans. Neural Netw.

(1999)

Cited by (13)

A novel quality-relevant fault detection method based on MICA-SOM multi-subspace partitioning for non-Gaussian industrial processes
2023, Journal of the Taiwan Institute of Chemical Engineers
Ensuring safe production and process quality is important for chemical processes. Therefore, a multi-subspace partitioning method based on modified independent component analysis (MICA) and self-organizing map (SOM) is proposed for quality-relevant fault detection. MICA can extract several independent components from the input data, where the mixing matrix implies relationship between independent components and input data.
According to the mixing matrix and SOM, the measured variables are classified into quality-relevant variables and quality-irrelevant variables. The quality-relevant variables are mapped into quality-relevant independent component space (Qr-ICS) and the quality-irrelevant independent component space (Qir-ICS) by modified independent component regression and feature decomposition. Then, Qr-ICS is used for quality-relevant fault detection. Quality-irrelevant variables are divided into several quality-irrelevant subspaces based on the correlations among variables. MICA models are established for the quality-irrelevant subspaces. Combined with the Qir-ICS, quality-irrelevant fault detection can be realized.
The application in the TE process shows that the proposed method has better quality-relevant monitoring reliability and quality-irrelevant monitoring performance than several comparison methods. The application in an actual industrial verifies the advantages and effectiveness of the proposed method over several traditional methods in practical applications.
Application of independent component analysis in regional flood frequency analysis: Comparison between quantile regression and parameter regression techniques
2020, Journal of Hydrology
Citation Excerpt :
Chen and Wang (2001) proposed a regression model where ICs are used as predictor variables in multiple linear regression (MLR) model and called it Independent Component Regression (ICR) technique. Numerous researchers have explored ICR in various fields of engineering; for example, Zhang et al. (2018) applied ICR for virtual sensing application, Kaneko et al. (2008) applied it to model aqueous solubility, and Lu et al. (2009) and Chowdhury et al. (2018) adopted it in financial forecasting. The applications of ICR are found mainly in signal processing and medical sciences; however, its application to hydrology is limited.
This paper examines the applicability of independent component analysis (ICA) in regional flood frequency analysis (RFFA). Data from 88 catchments in New South Wales, Australia are adopted in this study. The ICA is integrated with both quantile regression technique (QRT) and parameter regression technique (PRT). A total of eight climatic and catchment characteristics is used as potential predictor variables in the study. For ICA, independent components (IC) are used as the predictor variables and a ‘cumulative percent relevance’ criterion is adopted to select the best predictors. A leave-one-out validation is adopted to assess the performances of the competing models using a suit of statistical evaluation measures. It is found that the QRT model with four catchment characteristics, and the PR model with all the ICs as predictors outperform the other candidate models. These models provide a absolute median relative error in the range of 33% to 44% for the QRT model (with catchment characteristics as predictors), and 48% to 59% for the PRT model (with the ICs as predictors) for design flood estimation. This is comparable to the relative error values (57% to 64%) reported for the RFFA technique recommended in the Australian Rainfall and Runoff.
Prediction and causal analysis of defects in steel products: Handling nonnegative and highly overdispersed count data
2020, Control Engineering Practice
Citation Excerpt :
The present work focuses on predicting the number of defects in the steel plate and identifying the causal variable that may lead to defects. In the process industry, virtual sensing technologies or soft-sensors are a tool that can be used to predict product qualities or other important variables in real-time (Ge, Chen, & Song, 2011; Kadlec, Gabrys, & Strandt, 2009; Kano & Fujiwara, 2013; Ma & Huang, 2017; Souza, Araújo, & Mendes, 2016; Zhang, Kano, & Li, 2018). Soft-sensors can generally be divided into two categories, i.e., first-principle models and data-driven models.
In the steel industry, defects may occur during the manufacturing process. Thus, it is important to predict the occurrence of defects online in steel products and identify the causal variable that may lead to defects. However, the unique characteristics of the observed defect count data, such as nonnegative integers and high overdispersion, have posed some difficulties to the traditional probability models. To deal with this issue, the present work employs random forests to model and analyze the observed defect count data. Random forests are a nonlinear ensemble learning technique, which constructs several regression trees during the training phase and then predicts the output by averaging the predictions of each tree. Unlike the traditional probability models which are based on the specific distribution assumption, random forests are a non-parametric or distribution-free model. Furthermore, random forests can ensure the nonnegativity of the prediction, and thus it is suitable for defect count data modeling. In addition, partial dependence analysis in conjunction with the variable importance measure was used to identify the causal variable. The application results on the real steelmaking process have demonstrated that random forests outperform the PLS, SVR, Poisson, and NB methods in prediction accuracy. And the most influential variables identified by random forests are in line with operator experience.
Streaming parallel variational Bayesian supervised factor analysis for adaptive soft sensor modeling with big process data
2020, Journal of Process Control
Citation Excerpt :
Benefitting from the progress of computer science, data-driven models, which can be easily deployed in real industrial plants, has been widely utilized in process monitoring and soft sensor modeling [10,23,29,45,46]. In recent years, data-driven soft sensor is one of the most popular applications of data-driven models which provide real-time predictions for key quality variables that are difficult to measure in the process [24,42]. Data-driven models like principal component regression (PCR) [1], partial least squares (PLS) [31,33], artificial neural network (ANN) [32,39], kernel method [22,36] have been used for soft sensor modeling applications.
Time-varying and state shifting are two of the main process factors that cause poor prediction performance of soft sensors. Adaptive soft sensor is commonly an alternative practice to ensure high predictive accuracy. However, the large scale of process data often leads to inefficiency of model updating. In this paper, a streaming variational Bayesian supervised factor analysis (S-VBSFA) model is first proposed to capture the process time-varying and state shifting features through online updating of the posterior of model parameters. During the updating process, the symmetric Kullback–Leibler (SKL) divergence is utilized to determine priors of the next variation Bayesian inference. To improve the modeling efficiency for large-scale process data, the parallel computing strategy is further applied to the streaming model. As a result, the proposed streaming parallel VBSFA (SP-VBSFA) algorithm not only relieves the computing pressure of modeling big process data, but also improves the prediction accuracy and further reduces the tracking time delay for process variations. Two case studies demonstrate the superiority of the proposed method, compared to conventional methods.
Performance-relevant kernel independent component analysis based operating performance assessment for nonlinear and non-Gaussian industrial processes
2019, Chemical Engineering Science
Citation Excerpt :
Over the past decade, some criteria have been proposed for this issue from the perspective of process monitoring (Lee et al., 2006; Liu et al., 2008). Recently, Zhang et al. (2018) develop a quality-relevance contribution index and max-dependency to prioritize ICs for regression analysis. The pioneer work gives us a lot of inspiration and theoretical bases for our work.
The operating performance assessment of industrial processes becomes increasingly important in manufacturing production. A novel operating performance assessment method based on performance-relevant kernel independent component analysis is proposed here for nonlinear and non-Gaussian processes. The proposed method accounts for the comprehensive economic index, and the objectives are simultaneously to maximize the non-Gaussianity of independent components as well as the correlations between them and the comprehensive economic index. When applying it to online assessment, it demonstrates stronger robustness and higher sensitivity than the traditional methods do, which is attributed to its capacity in highlighting the performance-relevant variation information in modeling. Furthermore, both the performance grades and the conversions can be evaluated, which enhances the interpretability of the results. For the nonoptimality, the variable contributions are used to find the possible cause. Finally, the efficiency of the proposed method is illustrated by a case of gold hydrometallurgy process.
Distributed partial least squares based residual generation for statistical process monitoring
2019, Journal of Process Control
The main focus of the current work is to propose a purely data-based residual generation method for statistical process monitoring. The proposed approach utilizes but not limit to the partial least squares (PLS) algorithm to construct a specific regression model for each variable in a distributed manner, the model residual (i.e., estimation error) instead of the original data and the PLS latent components is then monitored. Given that every variable is transferred into the residual through its corresponding soft sensing model, the generated residual can reflect the variation in the defined input-output relationship. Furthermore, the residual is expected to follow a Gaussian distribution or at least much closer to a Gaussian distribution in contrast to the original data and the latent components, once the output variable is well predicted by the regression model. The main contributions of the presented work are as follows: 1) distributed soft sensing models for generating residuals, 2) statistical process monitoring for the generated residuals instead of original data, and 3) the comparison studies demonstrate the validity and superiority of the proposed monitoring scheme with the utilization of the PLS algorithm. It can be concluded from the comparisons and the illustrated superiority that the proposed approach would be an efficient and comparative alternative in process monitoring.

View all citing articles on Scopus

View full text

Quality-relevant independent component regression model for virtual sensing application

Highlights

Abstract

Introduction

Section snippets

Distance covariance and distance correlation

Quality-relevant independent component regression (QR-ICR)

Case studies

Conclusions

Chemom. Intell. Lab. Syst.

Neural Netw.

Comput. Chem. Eng.

Comput. Chem. Eng.

J. Process Control

J. Process Control

Comput. Chem. Eng.

IFAC-PapersOnLine

J. Process Control

Comput. Chem. Eng.

Talanta

J. Stat. Plan. Inference

Chemom. Intell. Lab. Syst.

Support vector machine regression (svr/ls-svm)-an alternative to neural networks (ann) for analytical chemistry? comparison of nonlinear methods on near infrared (nir) spectroscopy data

Analyst

Nonlinear Programming

A new approach to near-infrared spectral data analysis using independent component analysis

J. Chem. Inf. Comput. Sci.

Multi-objective optimization

Search Methodologies

Exploratory projection pursuit

J. Am. Stat. Assoc.

Independent component analysis yields chemically interpretable latent variables in multivariate regression

J. Chem. Inf. Model.

Fast and robust fixed-point algorithms for independent component analysis

IEEE Trans. Neural Netw.