Bayesian-based on-line applicability evaluation of neural network models in modeling automotive paint spray operations
Introduction
Neural network (NN) techniques have been widely used in system modeling, information classification, process control and optimization in numerous industrial applications (Meireles, Almeida, & Simoes, 2003). An NN is usually structured with connected nodes in a certain topology. In model development, the weights of the connections are adjusted by an algorithm in training, which is a nonlinear optimization process (Dayholf, 1992). Numerous NN training/validation algorithms and tools are available that are effective in generating superior models (Hamm, Brorsen, & Hagan, 2002; Huang, Edgar, Himmelblau, & Trachtenberg, 1994; Kwok & Yeung, 1997; Mackey, 1992a, Mackey, 1992b; Porto, Fogel, & Fogel, 1995; Reed, 1993). A well trained/validated NN should extract sufficient knowledge that is hidden in a pool of quality data; it should be then capable of using the knowledge to characterize adequately the system of interest. It is always expected that an NN be reliable in terms of prediction accuracy and generalization capability in application (Alippi, 2002; Levin, Tishby, & Solla, 1990).
A major challenge of model application is how to ensure prediction accuracy continuously. In general, the training and validation data can never be perfect. On the other hand, industrial systems always experience various disturbances and fluctuations, possibly not conceivable, due to a variety of reasons. Hence, the system behavior demonstrated under these circumstances may not be fully captured by the adopted single model based on a specific criterion used in training (Yao, 1999). Some nearly equally good, but discarded models, either topology-wise or parameter-wise different from the adopted one, could have been otherwise chosen if a model selection criterion was different. These models, trained/validated using the same data as that for the adopted model, may demonstrate better prediction and generalization capability under certain circumstances in application. It is argued, therefore, that it might be of advantages for adopting multiple models for application; certainly, these models should be all superior with respect to the criteria used in model development. Petridis and Kehagias, 1996, Petridis and Kehagias, 1998 and Petridis et al. (2001) suggested using multiple models to produce predictions based on their weighted outputs. In this regard, a major challenge is changed from how to identify the most suitable model, which is always arguable, to how to use a set of nearly equality superior models properly (Busemeyer & Wang, 2000; Forster, 2000).
One of the key tasks of using a model set is to determine how to evaluate the applicability of each model under each specific condition; it is a model-set management issue. It is conceivable that the applicability of each model may be time dependent, as system environment changes along the time. Thus, the applicability of each model should be periodically evaluated for the most suitable use.
In this paper, a model-set-based system characterization scheme is described first. Then, a Bayesian-based model-set management method is introduced for constructing a statistically superior model set for on-line application. Specifically, this method is for evaluating each model in the model set based on each new application environment, which is described by newly available system data. Overall system characterization is based on weighted model predictions. A repeated use of the method keeps the weights updated constantly based on newly available system data. This can make the model-set-based system performance description more precise and robust. To automate the model weight update in on-line application, a Bayesian-based method application procedure will be also introduced. The efficacy of the method will be demonstrated by a case study on automotive thin film thickness prediction by a set of structurally and/or parametrically different NN models.
Section snippets
Multiple model-based system prediction
Given a model set, M, which contains N well-trained and validated models:Each model has the same type of inputs, X ∈ Rp, and outputs, Y ∈ Rq, as they are for the same application, but their structures and/or parameters are different. At time tk, if the following set of input data is available:Then, the N models can be used to generate the following output predictions:where is the prediction of system behavior by
Bayesian method basics
Baye's theory-based methods use probabilities to describe the degree of belief in parameter values or models, and can give statistical results based on known data (Hoeting, Madigan, Raftery, & Volinsky, 1999; Vila, Wagner, & Neveu, 2000; Wasserman, 2000). The methods have been adopted in a wide array of parameter identification or model comparison approaches in either linear or nonlinear regression modeling (Denison, Holmes, Mallick, & Smith 2002; Raftery, Madigan, & Hoeting, 1997). In data
Bayesian-based model applicability assessment
According to the analysis of Eq. (11), the posterior probability of a model can be considered as a preference of the model. Note that the model preference may change along the time because a model prediction performance may be different. Thus, the weights assigned to the models may change along the time. The engineering basis for weight change is established by viewing new application situations, which are reflected by those newly collected data, and by examining the application performance of
Automated model-set management procedure
The Bayesian-based weight determination method described in the preceding section can be used to evaluate periodically the degree of applicability of the models in the model set and to create a weighted average prediction for system characterization. The method can be embedded in the following procedure for automatic, continuous management of the model set.
- •
Step 1. Generate model set M that contains N well-trained and validated models (see, Eq. (1)). The models can be structurally or
Case study
In automotive paint shops, quality control (QC) of vehicle coating has been traditionally realized in an open-loop fashion, i.e., the quality is controlled through inspection on a final product. It is proven that such type of QC is passive methodologically. Filev (2002) introduced a closed-loop QC approach by which a Jacobian matrix model is used in film thickness closed-loop control.
To describe the nonlinear, multi-variable process more comprehensively, a number of NN's have been introduced to
Discussion
For a successful use of the model-set management procedure, two issues need to be discussed. The first is about how to use the procedure when more than one set of data is available each time when the model weights are to be re-evaluated, and the second is how to select a value of deviation σ that is directly related to the calculation of the weights (see, Eq. (25a), (25b)).
Concluding remarks
In model-based system characterization, the use of a model set may have advantages against the use of a single model. Usually, a model-set-based prediction should be more precise, smoother and more robust, if the models in the model set are well trained and validated and they are managed properly. The difficulty of model set management is the determination of the acceptance of the prediction by each individual model in system characterization. In this paper, a Bayesian-based weight
Acknowledgments
This work is in part supported by NSF (CTS 0091398) and the Institute of Manufacturing Research of Wayne State University.
References (29)
- et al.
Model comparisons and model selections based on generalization criterion methodology
Journal of Mathematical Psychology
(2000) Key concepts in model selection: Performance and generalizability
Journal of Mathematical Psychology
(2000)- et al.
A multi-model algorithm for parameter estimation of time varying nonlinear systems
Automatica
(1998) Bayesian model selection and model averaging
Journal of Mathematical Psychology
(2000)Selecting accurate, robust, and minimal feedforward neural networks
IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications
(2002)- et al.
Bayesian theory
(1994) Neural network architectures
(1992)- et al.
Bayesian methods for nonlinear classification and regression
(2002) - Filev, D. P. (2002). Applied intelligent control—Control of automotive paint process. In Proceedings of the 2002 world...
- Hamm, L., Brorsen, B. W., & Hagan, M. T. (2002). Global optimization of neural network weights. In Proceedings of the...
Bayesian model averaging: A tutorial
Statistical Science
Constructing a reliable neural network model for a plasma etching process using limited experimental data
IEEE Transactions on Semiconductor Manufacturing
Constructive algorithms for structure learning in feedforward neural networks for regression problems
IEEE Transactions on Neural Networks
Bayesian approach for neural networks—Review and case studies
Neural Networks
Cited by (6)
On-line Bayesian-based Model-set Management Method with Case Study of Steam Reforming Prediction under Various Feed Compositions
2018, Computer Aided Chemical EngineeringPrediction of product formation in 2-keto-L-gulonic acid fermentation through Bayesian combination of multiple neural networks
2014, Process BiochemistryCitation Excerpt :BCP is presented as a probabilistically motivated predictor for time series prediction. By incorporating an appropriate number of specialized predictors, the combination module can activate the appropriate predictor at the appropriate time, and therefore yield superior performance [18–23]. In this study, the prediction of product formation is achieved by Bayesian combination of three neural networks.
Design of inferential sensors in the process industry: A review of Bayesian methods
2013, Journal of Process ControlCitation Excerpt :The authors applied their proposed method to developing an adaptive multi-model inferential sensor for real-time monitoring of a key quality variable in an industrial oil sands processing unit. Li and Huang [152] introduced a Bayesian-based model-set management method for selecting a statistically superior model-set for implementation of multi-model inferential sensors. The authors applied their method to design of a multi-model inferential sensors for automotive paint spray process where the thickness of the thin film on the vehicle surface should be precisely predicted.
Integrated multiscale product and process control of polymeric coating curing
2010, Chemical Engineering JournalModeling and prediction of paint film deposition rate for robotic spray painting
2011, 2011 IEEE International Conference on Mechatronics and Automation, ICMA 2011Proactive quality paint thickness control using ANFIS
2010, Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics