Application of artificial neural networks in linear profile monitoring

doi:10.1016/j.eswa.2010.09.160

Expert Systems with Applications

Volume 38, Issue 5, May 2011, Pages 4920-4928

https://doi.org/10.1016/j.eswa.2010.09.160 Get rights and content

Abstract

In many quality control applications the quality of process or product is characterized and summarized by a relation (profile) between a response variable and one or more explanatory variables. Such profiles can be modeled using linear or nonlinear regression models. In this paper we use artificial neural networks to detect and classify the shifts in linear profiles. Three monitoring methods based on artificial neural networks are developed to monitor linear profiles. Their efficacies are assessed using average run length criterion.

Research highlights

► Three ANN based methods for monitoring linear profiles are proposed. ► Proposed methods perform better than T², EWMA/R and EWMA-3 charts for medium to large shifts. ► A advantage of proposed ANN methods is to train the neural networks to detect desired shifts. ► ARL criterion is used to assess the efficiencies of the methods. ► Control charts are used to detect shifts in intercept, slope and residuals.

Introduction

Statistical process control (SPC) has been successfully applied in a variety of areas, especially in industries, since Shewhart introduced the control chart in 1924. To monitor a process, two different approaches may be used: (1) determine the process state using the distribution function of a single or multiple quality characteristics; (2) to use a profile or a function of the quality characteristic(s). For the former case, different control charts have been proposed, of which some are univariate such as Shewhart, Cumulative SUM (CUSUM) and EWMA, and some such as T², MCUSUM and MEWMA are multivariate (for details see Montgomery (2009)). For the latter approach, which has been given more attention in recent literature, some researchers used different terminologies to express the profile. Gardner et al. (1997) applied the term “signature” in their study while Jin and Shi (2001) used “waveform signals”. The signals, such as tonnage signals in the stamping process, torque signals in the tapping, and force signals in the welding process, are collected by the sensors during production processes.

In profile monitoring, it is assumed that for each profile jth (j > 1) value of the response variable (Y) is measured along with the corresponding values of one or more explanatory variables (the Xs), reflecting the location of the measurement on a process. Kang and Albin (2000) gave two examples of a process characterized by a profile; one is in a semiconductor manufacturing quality control problem in which the performance of the mass flow controller is monitored by a linear function. In another example, there was mention of monitoring aspartame (an artificial sweetener) by a profile. Moreover, Jin and Shi, 1999, Jin and Shi, 2001 applied linear profile monitoring to monitor stamping tonnage and waveform.

Literature on profile monitoring has addressed both linear profile monitoring and nonlinear profiles in phase I and phase II. Williams, Woodall, Spitzner, Montgomery, and Gupta (2004) discussed some of the general issues in using control charts to monitor linear and nonlinear profiles. Kang and Albin (2000) proposed two approaches for monitoring linear profiles; using multivariate T² chart introduced by Hotelling in 1947, and examining the residuals by using EWMA and R charts. Mahmoud and Woodall (2004) have discussed phase I of monitoring simple linear profile while samples are collected over time; a situation common in calibration applications. Kim, Mahmoud, and Woodall (2003) proposed alternative control charts for monitoring profile using estimated regression coefficients from each sample to construct three separate EWMA charts. Zou, Tsung, and Wang (2007) proposed a self-starting control chart based on recursive residuals in phase I and II of linear profile. Mahmoud, Parker, Woodall, and Hawkins (2007) presented a method to detect change point in linear profile monitoring using segmented regression technique. Zhang, Li, and Wang (2009) integrated likelihood ratio and EWMA control chart to monitor linear profile in phase II. Saghaei, Mehrjoo, and Amiri (2009) used CUSUM control chart in monitoring linear profile. Noorossana, Amiri, and Soleimani (2008) investigated the effect of autocorrelation between observations in each linear profile and used three methods based on time series to eliminate autocorrelation before monitoring the process. Walker and Wright (2002) used models named additive models to represent the curves of interest in vertical density profiles in monitoring of particle board. Miller (2002) applied linear and nonlinear types of response functions in design experiments. As profile monitoring falls under the broad field of functional data analysis, Ramsay and Silverman (1997) have discussed various examples of functional data or profiles. Brill (2001) applied T² to monitor the coefficients of a nonlinear regression function in a chemical process. Williams, Woodall, and Berch (2003) studied the use of T² control chart to monitor the coefficients of a nonlinear regression fitted to successive sets of profile data.

In this paper we deploy neural networks to detect and classify shifts in the linear profiles. We propose three methods that use neural networks to monitor linear profiles. In the first and second methods we present a single neural network to detect shifts in profile. In the third one, we propose using three neural networks simultaneously to detect and classify profile shifts. In Section 2, we provide a brief background on the linear regression models, T², EWMA/R and EWMA-3 control charts. These methods are used in our comparison study. In Section 3, an introduction to neural network and its application in statistical process control are presented. In Section 4, we discuss the application of the neural networks in monitoring linear profiles. Section 5 presents the comparison study between the performances of the proposed methods, using T², EWMA/R and EWMA-3 control charts. We have used the numerical example given by Kang and Albin (2000) and deployed simulation to calculate average run length (ARL) of our proposed methods. Finally, in Section 6, we conclude with some recommendations for future work in this area.

Section snippets

Background

Here we provide some brief backgrounds on the simple linear regression models. Then we describe the applications of T², EWMA/R and EWMA-3 control charts in profile monitoring.

Neural networks and their application in statistical process control

The neural network used in this paper is a multilayer perceptron neural network (MLP), which implements the Back-Propagation training rule developed by Rumelhart, McClelland, and PDP Research Group (1986). In what follows, MLP and the corresponding training algorithms are briefly reviewed.

Application of the perceptron neural networks to monitor linear profiles

In using PNN, we model shift detections in profile monitoring as a classification problem. Hence, we expect PNN to classify in-control data from out-of-control data. Three methods are developed in this paper to monitor linear profiles. In the first method, we design a PNN to detect shifts in slope, intercept and variance of residuals by using estimated values of the regression coefficients and sample means. We named this method ANN1. The second method is similar to the first one, but we

Evaluation and comparison study

In this section, we use the example given by Kang and Albin (2000) to evaluate the performance of our proposed methods. The comparison study with EWMA-3, EWMA/R and T² methods is also presented. All considered charts are designed to have the same overall in-control ARL of 200. In this simulation study, a total of 10,000 replications were used to estimate ARL value for each chart. The underlying in-control linear profile model used by Kang and Albin (2000) is y_ij = 3 + 2x_i + ε_ij where the ε_ij’s are

Conclusion and recommendations for future research

In this paper we employed neural networks to propose three methods for monitoring linear profiles. We have used ARL criterion to assess the efficiencies of the proposed methods. Using simulation study we then compared the ARLs of the proposed methods with the corresponding ARLs of T², EWMA/R and EWMA-3 charts when detecting sustained shifts in y-intercept and slope or increases in the residual variance. The results show that ANN-a and ANN3 methods had better performances than T² chart

References (34)

B. Abbasi
A neural network applied to estimate process capability of non-normal processes
Expert Systems with Applications
(2009)
G.A. Pugh
Synthetic neural networks for process control
Computers and Industrial Engineering
(1989)
G.A. Pugh
A comparison of neural networks to SPC charts
International Journal of Production Research
(1991)
J. Zhang et al.
Control chart based on likelihood ratio for monitoring linear profiles
Computational Statistics and Data Analysis
(2009)
C. Bishop
Neural networks for pattern recognition
(1995)
Brill, R. V. (2001). A case study for control charting a product quality measure that is a continuous function over...
S.I. Chang et al.
A two-stage network approach for process variance change detection and classification
International Journal of Production Research
(1999)
S.V. Crowder et al.
An EWMA for monitoring a process standard deviation
Journal of Quality Technology
(1992)
M.M. Gardner et al.
Equipment fault detection using spatial signatures
IEEE Transactions on Components, Packaging, and Manufacturing Technology – Part C
(1997)
S. Haykin
Neural networks: A comprehensive foundation
(1994)

H.B. Hwarng et al.

X-bar control chart pattern identification through efficient off-line neural network training

IIE Transactions

(1993)

J. Jin et al.

Feature-preserving data compression of stamping tonnage information using wavelets

Technometrics

(1999)

J. Jin et al.

Automatic feature extraction of waveform signals for in-process diagnostic performance improvement

Journal of Intelligent Manufacturing

(2001)

L. Kang et al.

On-line monitoring when the process yields a linear profile

Journal of Quality Technology

(2000)

K. Kim et al.

On the monitoring of linear profiles

Journal of Quality Technology

(2003)

C.A. Lowry et al.

The performance of control charts for monitoring process dispersion

Communications in Statistics – Simulation and Computation

(1995)

M.A. Mahmoud et al.

A change point method for linear profile data

Quality and Reliability Engineering International

(2007)

Cited by (40)

New statistical and machine learning based control charts with variable parameters for monitoring generalized linear model profiles
2023, Computers and Industrial Engineering
In this research, we develop three statistical based control charts: the Hotelling’s T², MEWMA (multivariate exponentially weighted moving average), and LRT (likelihood ratio test) as well as three machine learning (ML) based control charts: the ANN (artificial neural network), SVR (support vector regression), and RFR (random forest regression), for monitoring generalized linear model (GLM) profiles. We train these ML models with two different training methods to get a linear (regression) output and then apply our classification technique to see if the process is in-control or out-of-control, at each sampling time. In addition to developing the FP (fixed parameters) schemes, for the first time in GLM profiles, we design an adaptive VP (variable parameters) scheme for each control chart as well to increase the charts’ sensitivity in detecting shifts. We develop some algorithms with which the values of the control chart parameters in both FP and VP schemes can be obtained. Then, we develop two algorithms to measure the charts’ performance in both FP and VP schemes, by using the run-length and time-to-signal based performance measures. This is also the first control chart-related research that develops an algorithm to compute the performance measures that applies to any VP adaptive control scheme.
After designing the control charts as well as performance measures, we perform extensive simulation studies and evaluate and compare all our control charts under different shift sizes and scenarios, and in three different simulation environments. Finally, we present a numerical example regarding a drug dose-response study to show how the proposed control charts can be implemented in real practice.
A network surveillance approach using machine learning based control charts
2023, Expert Systems with Applications
Network surveillance, i.e., the detection of anomalous behaviour in communications in a network, has become an important issue in recent years. In this field, techniques of statistical process monitoring, especially control charts, have been frequently applied to monitor the probability of existing connections between the members (nodes) of such a network. As another approach, monitoring the number of communications between the nodes is also an important way to identify network anomalies. Previous works have focused on conventional statistical control charts, but there are no machine learning approaches, although the performance of machine learning based control charts has been shown to be excellent in several other applications. This paper aims to develop machine learning based control charts for network surveillance problems in the case of monitoring the number of communications between the nodes. The results of extensive Monte Carlo simulations reveal the superiority of the proposed method over conventional competitors.
An ensemble neural network framework for improving the detection ability of a base control chart in non-parametric profile monitoring
2022, Expert Systems with Applications
Profile monitoring is a challenging issue in statistical process control (SPC). It aims to use a functional relationship between a response variable and one or more explanatory variable(s) to summarize the quality of a process/product. Most of the existing studies consider the same form of a functional relationship for both in-control (IC) and out-of-control (OC) situations or parametric approaches. However, non-parametric profiles with different relationships in OC conditions are very common. In this paper, we propose a novel ensemble framework to monitor changes in both regression relationship and variation of the profile for Phase II applications. This approach employs a pool of artificial neural networks (ANNs) as learners to enhance the efficiency of a base control chart, which is a non-parametric exponentially weighted moving average (NEWMA) in this study. Then, a further ANN is used as a reasoning scheme (incorporator) to perform prediction by combining the outcomes of the learners. Experimental results demonstrate the effectiveness of the proposed framework, denoted by EANNN, in comparison with the base control chart, i.e., NEWMA, and other non-parametric methods. In addition, a practical example regarding a deep reactive ion-etching process from semiconductor device fabrication is provided to show the implementation of the proposed method.
Using evolutionary artificial neural networks in monitoring binary and polytomous logistic profiles
2021, Journal of Manufacturing Systems
Citation Excerpt :
Maybe the reason of the reluctance for employing these techniques by researchers is because of inability to compete with statistical approaches. For example, the three proposed ANNs in Hosseinifard, Abdollahian and Zeephongsekul [32] was not able to acquire remarkable results for simple linear profiles, so the efficiency of this methods was enhanced in Yeganeh and Shadman [33] by using of some novel run-rules. Although they reached competitive results for simple linear profiles with these run-rules, the structure of ANN was not tangibly changed, expect some modification of input layer.
A statistical profile is a relationship between a quality characteristic (a response) and one or more explanatory variables to characterize quality of a process or a product. Monitoring profiles or checking the stability of profiles over time, has been extensively studied under the normal response variable, but it has paid a little attention to the profile with the non-normal response variable denoted by generalized linear models (GLM). Whereas, some of the potential applications of profile monitoring are cases where the response can be modelled using logistic profiles entailing binary, nominal and ordinal models. Also, most of existing control charts in this field have been developed by statistical approach and employing machine learning techniques have been rarely addressed in the related literature. Hence, to implement on-line process monitoring of logistic profiles, a novel artificial neural network (ANN) as a control chart with a heuristic training procedure is proposed in this paper. Performance of the proposed approach is investigated and compared using simulation studies in binary and polytomous models based on average run length (ARL) criterion. Simulation results revealed a good performance of the proposed approach. Nevertheless, to enhance the detection ability of the proposed approach more, the idea of combining run-rule which is a supplementary tool for making more sensitive control chart with final statistic is also implemented in this paper. Furthermore, a diagnostic method with machine learning schemes is employed to identify the shifted parameters in the profile. Results indicate the superior performance of the proposed approaches in most of the simulations. Finally, an example is used to illustrate the implementation of the proposed charting scheme.
Monitoring logistic profiles using variable sample interval approach
2021, Computers and Industrial Engineering
Citation Excerpt :
On the other hand, support vector regression (SVR) as a machine learning technique is developed for monitoring logistic profiles in phase II. It was inspired by the idea of Hosseinifard, Abdollahian, and Zeephongsekul (2011), Huwang, Wang, Xue, and Zou (2014) and Yeganeh and Shadman (2020)’s papers. In the mentioned papers, artificial neural network (ANN) was trained as a control chart in monitoring simple linear profiles.
Investigation the stability of the relationship between a response variable and one or more independent variables over time is termed as profile monitoring. In the area of profile monitoring, fewer researches can be found to the profiles with the non-normal response variable denoted by generalized linear models (GLM). Logistic profiles as one of common the GLM profiles, are the most common types of profiles in which we take advantage of logistic regression to model the relationship. In logistic regression, the response variable is assumed to be non-quantitative and binary. Also, to increase the efficiency of the charts and faster detection of the shifts, we can consider control charts’ parameters such as sample size, sampling intervals and etc., with an adaptive approach, which means that considering the process conditions, parameter values can be unfixed and variable. In this study, we investigate logistic profiles monitoring in phase II using three existing control charts entailing T² control charts, multivariate exponentially weighted moving average (MEWMA), deviance control chart and two proposed control chart including generalized likelihood ratio (GLR) and support vector regression (SVR) methods by considering sampling interval as a variable, denoted with variable sample interval (VSI) approach. Simulation results indicate the much better performance of control charts by adding the VSI than the traditional fixed sampling interval in terms of average time to signal (ATS). Eventually, to clarify how to implement the VSI charts, a numerical example is provided.
Monitoring linear profiles using Artificial Neural Networks with run rules
2021, Expert Systems with Applications
Citation Excerpt :
The difference between these two networks is in the intercept (for the first is 3 and second 13) and explanatory variables (for the first is 2:2:8 and second −3:2:3). In the next sections, our proposed ANNWR1 and ANNWR2 control charts are compared with the methods proposed by Kang and Albin (2000), Kim et al. (2003), Zou et al. (2007), Saghaei et al. (2009), Zhang et al. (2009), Hosseinifard et al. (2011), Xu et al. (2012), Huwang et al. (2014), Motasemi et al. (2017) and Saeed et al. (2018) (hereinafter referred to as EWMAR and T2 (both for Kang & Albin, 2000), EWMA3, MEWMA, LRT, CUSUM3, ANN1, GLR, VSCS, EWMA-AREA, PM-3, respectively). Note that all the results are based on 10,000 simulation runs; in each shift, we try to compare our methods with the best methods in the context of profile monitoring (the methods with lowest ARL1).
In some applications, a relation between a response variable and one or more explanatory variables (referred as a “profile”) characterizes the quality of a process. Profile monitoring is commonly performed through statistical methods, while machine learning schemes have not received much attention in this regard. In this paper, a control chart based on Artificial Neural Networks (ANN) is proposed to monitor linear profiles in phase II. In the proposed control chart, some novel run rules as the major contribution of this paper are also used to enhance the efficiency of the control chart and for faster detection of shifts. Simulation results revealed a good performance of the proposed control chart based on average run length (ARL) criterion. Further, a systematic ANN-based diagnostic procedure was proposed to identify which parameter has changed in the process. Finally, the implementation of the proposed scheme was illustrated through a real calibration example from the field of chemical engineering.

View all citing articles on Scopus

View full text

Application of artificial neural networks in linear profile monitoring

Abstract

Research highlights

Introduction

Section snippets

Background

Neural networks and their application in statistical process control

Application of the perceptron neural networks to monitor linear profiles

Evaluation and comparison study

Conclusion and recommendations for future research

Expert Systems with Applications

Computers and Industrial Engineering

International Journal of Production Research

Computational Statistics and Data Analysis

Neural networks for pattern recognition

A two-stage network approach for process variance change detection and classification

International Journal of Production Research

An EWMA for monitoring a process standard deviation

Journal of Quality Technology

Equipment fault detection using spatial signatures

IEEE Transactions on Components, Packaging, and Manufacturing Technology – Part C

Neural networks: A comprehensive foundation

X-bar control chart pattern identification through efficient off-line neural network training

IIE Transactions

Feature-preserving data compression of stamping tonnage information using wavelets

Technometrics

Automatic feature extraction of waveform signals for in-process diagnostic performance improvement

Journal of Intelligent Manufacturing

On-line monitoring when the process yields a linear profile

Journal of Quality Technology

On the monitoring of linear profiles

Journal of Quality Technology

The performance of control charts for monitoring process dispersion

Communications in Statistics – Simulation and Computation

A change point method for linear profile data

Quality and Reliability Engineering International