Bayesian-based on-line applicability evaluation of neural network models in modeling automotive paint spray operations

https://doi.org/10.1016/j.compchemeng.2006.03.005Get rights and content

Abstract

The neural network (NN) models well trained and validated by the same data may exhibit noticeably different predictabilities in applications. This is mainly due to the fact that the knowledge captured by the NNs in training may be different in depth and breadth. In this regard, using a set of nearly equally superior models, instead of a single one, may demonstrate its robustness of system performance prediction in on-line application. An unresolved issue, then, is how to value the prediction by each model of the model set in each application step.

In this paper, we introduce a Bayesian-based model-set management method for constructing a statistically superior model set for on-line application. Specifically, this method is for manipulating the model set by assigning statistically most appropriate weights to the model predictions; the weighted model predictions determine overall system behavior. A repeated use of the method keeps the weights updated constantly based on the newly available system data, which makes the model-set-based system performance description more precise and robust. The efficacy of the method is demonstrated by studying an automotive paint spray process where the thin film thickness on vehicle surface should be precisely predicted.

Introduction

Neural network (NN) techniques have been widely used in system modeling, information classification, process control and optimization in numerous industrial applications (Meireles, Almeida, & Simoes, 2003). An NN is usually structured with connected nodes in a certain topology. In model development, the weights of the connections are adjusted by an algorithm in training, which is a nonlinear optimization process (Dayholf, 1992). Numerous NN training/validation algorithms and tools are available that are effective in generating superior models (Hamm, Brorsen, & Hagan, 2002; Huang, Edgar, Himmelblau, & Trachtenberg, 1994; Kwok & Yeung, 1997; Mackey, 1992a, Mackey, 1992b; Porto, Fogel, & Fogel, 1995; Reed, 1993). A well trained/validated NN should extract sufficient knowledge that is hidden in a pool of quality data; it should be then capable of using the knowledge to characterize adequately the system of interest. It is always expected that an NN be reliable in terms of prediction accuracy and generalization capability in application (Alippi, 2002; Levin, Tishby, & Solla, 1990).

A major challenge of model application is how to ensure prediction accuracy continuously. In general, the training and validation data can never be perfect. On the other hand, industrial systems always experience various disturbances and fluctuations, possibly not conceivable, due to a variety of reasons. Hence, the system behavior demonstrated under these circumstances may not be fully captured by the adopted single model based on a specific criterion used in training (Yao, 1999). Some nearly equally good, but discarded models, either topology-wise or parameter-wise different from the adopted one, could have been otherwise chosen if a model selection criterion was different. These models, trained/validated using the same data as that for the adopted model, may demonstrate better prediction and generalization capability under certain circumstances in application. It is argued, therefore, that it might be of advantages for adopting multiple models for application; certainly, these models should be all superior with respect to the criteria used in model development. Petridis and Kehagias, 1996, Petridis and Kehagias, 1998 and Petridis et al. (2001) suggested using multiple models to produce predictions based on their weighted outputs. In this regard, a major challenge is changed from how to identify the most suitable model, which is always arguable, to how to use a set of nearly equality superior models properly (Busemeyer & Wang, 2000; Forster, 2000).

One of the key tasks of using a model set is to determine how to evaluate the applicability of each model under each specific condition; it is a model-set management issue. It is conceivable that the applicability of each model may be time dependent, as system environment changes along the time. Thus, the applicability of each model should be periodically evaluated for the most suitable use.

In this paper, a model-set-based system characterization scheme is described first. Then, a Bayesian-based model-set management method is introduced for constructing a statistically superior model set for on-line application. Specifically, this method is for evaluating each model in the model set based on each new application environment, which is described by newly available system data. Overall system characterization is based on weighted model predictions. A repeated use of the method keeps the weights updated constantly based on newly available system data. This can make the model-set-based system performance description more precise and robust. To automate the model weight update in on-line application, a Bayesian-based method application procedure will be also introduced. The efficacy of the method will be demonstrated by a case study on automotive thin film thickness prediction by a set of structurally and/or parametrically different NN models.

Section snippets

Multiple model-based system prediction

Given a model set, M, which contains N well-trained and validated models:M={m1,m2,,mN}Each model has the same type of inputs, X  Rp, and outputs, Y  Rq, as they are for the same application, but their structures and/or parameters are different. At time tk, if the following set of input data is available:Dx(tk)={x1(tk),x2(tk),,xp(tk)}Then, the N models can be used to generate the following output predictions:Yˆ(tk)={yˆ1(tk),yˆ2(tk),,yˆN(tk)}where yˆi(tk) is the prediction of system behavior by

Bayesian method basics

Baye's theory-based methods use probabilities to describe the degree of belief in parameter values or models, and can give statistical results based on known data (Hoeting, Madigan, Raftery, & Volinsky, 1999; Vila, Wagner, & Neveu, 2000; Wasserman, 2000). The methods have been adopted in a wide array of parameter identification or model comparison approaches in either linear or nonlinear regression modeling (Denison, Holmes, Mallick, & Smith 2002; Raftery, Madigan, & Hoeting, 1997). In data

Bayesian-based model applicability assessment

According to the analysis of Eq. (11), the posterior probability of a model can be considered as a preference of the model. Note that the model preference may change along the time because a model prediction performance may be different. Thus, the weights assigned to the models may change along the time. The engineering basis for weight change is established by viewing new application situations, which are reflected by those newly collected data, and by examining the application performance of

Automated model-set management procedure

The Bayesian-based weight determination method described in the preceding section can be used to evaluate periodically the degree of applicability of the models in the model set and to create a weighted average prediction for system characterization. The method can be embedded in the following procedure for automatic, continuous management of the model set.

  • Step 1. Generate model set M that contains N well-trained and validated models (see, Eq. (1)). The models can be structurally or

Case study

In automotive paint shops, quality control (QC) of vehicle coating has been traditionally realized in an open-loop fashion, i.e., the quality is controlled through inspection on a final product. It is proven that such type of QC is passive methodologically. Filev (2002) introduced a closed-loop QC approach by which a Jacobian matrix model is used in film thickness closed-loop control.

To describe the nonlinear, multi-variable process more comprehensively, a number of NN's have been introduced to

Discussion

For a successful use of the model-set management procedure, two issues need to be discussed. The first is about how to use the procedure when more than one set of data is available each time when the model weights are to be re-evaluated, and the second is how to select a value of deviation σ that is directly related to the calculation of the weights (see, Eq. (25a), (25b)).

Concluding remarks

In model-based system characterization, the use of a model set may have advantages against the use of a single model. Usually, a model-set-based prediction should be more precise, smoother and more robust, if the models in the model set are well trained and validated and they are managed properly. The difficulty of model set management is the determination of the acceptance of the prediction by each individual model in system characterization. In this paper, a Bayesian-based weight

Acknowledgments

This work is in part supported by NSF (CTS 0091398) and the Institute of Manufacturing Research of Wayne State University.

References (29)

  • J.A. Hoeting et al.

    Bayesian model averaging: A tutorial

    Statistical Science

    (1999)
  • Y.L. Huang et al.

    Constructing a reliable neural network model for a plasma etching process using limited experimental data

    IEEE Transactions on Semiconductor Manufacturing

    (1994)
  • T.Y. Kwok et al.

    Constructive algorithms for structure learning in feedforward neural networks for regression problems

    IEEE Transactions on Neural Networks

    (1997)
  • J. Lampinen et al.

    Bayesian approach for neural networks—Review and case studies

    Neural Networks

    (2001)
  • Cited by (6)

    • Prediction of product formation in 2-keto-L-gulonic acid fermentation through Bayesian combination of multiple neural networks

      2014, Process Biochemistry
      Citation Excerpt :

      BCP is presented as a probabilistically motivated predictor for time series prediction. By incorporating an appropriate number of specialized predictors, the combination module can activate the appropriate predictor at the appropriate time, and therefore yield superior performance [18–23]. In this study, the prediction of product formation is achieved by Bayesian combination of three neural networks.

    • Design of inferential sensors in the process industry: A review of Bayesian methods

      2013, Journal of Process Control
      Citation Excerpt :

      The authors applied their proposed method to developing an adaptive multi-model inferential sensor for real-time monitoring of a key quality variable in an industrial oil sands processing unit. Li and Huang [152] introduced a Bayesian-based model-set management method for selecting a statistically superior model-set for implementation of multi-model inferential sensors. The authors applied their method to design of a multi-model inferential sensors for automotive paint spray process where the thickness of the thin film on the vehicle surface should be precisely predicted.

    • Modeling and prediction of paint film deposition rate for robotic spray painting

      2011, 2011 IEEE International Conference on Mechatronics and Automation, ICMA 2011
    • Proactive quality paint thickness control using ANFIS

      2010, Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
    View full text