Cope with diverse data structures in multi-fidelity modeling: A Gaussian process method
Introduction
In recent years, new advances in computers and computing science lead to the widespread use of computer simulation models, e.g., computational fluid dynamics (CFD) and finite element analysis (FEA), in engineering design and optimization. In simulation based engineering problems, surrogates are starting to play an important role, since they can approximate the expensive simulation model at some training points for alleviating computational burden. Gaussian process regression (Rasmussen and Williams, 2006), also known as Kriging (Lophaven et al., 2002), is a widely used surrogate model, since it can provide not only the prediction response but also the related prediction variance.
This article focuses on a multi-fidelity scenario where the simulator for the physics-based problem of interests can be run at multiple levels of fidelity. The high fidelity (HF) simulator yields the most accurate predictions but is most time-consuming; whereas the fast low fidelity (LF) simulators provide coarse predictions, which however include the main features of the problem and thus are useful for preliminary exploration. The LF simulators are usually simplified analysis models by using coarse finite element meshes, relaxed boundary or convergence conditions, etc. For example, it was reported by Benamara et al. (2016) that the HF simulation for a 1.5 stage booster has 5 million meshes and requires 2 h; but the LF simulation has only 0.7 million meshes and requires only 15 min. In practice, we cannot afford extensive HF simulations at many training points but many LF simulations are affordable. Suppose that the simulator has Q levels of fidelity, the multi-fidelity modeling (MFM), also known as variable-fidelity modeling or data fusion, attempts to utilize the knowledge from the correlated yet inexpensive LF simulators to enhance the modeling of the expensive HF simulator.
Considering levels of fidelity as correlated outputs, the information fusion can be achieved in the multi-output modeling framework. The multi-output GP (MOGP), also known as multi-variate Kriging (Kleijnen and Mehdad, 2014), has been developed and investigated with a long history. The MOGP attempts to model multiple correlated outputs simultaneously by sharing the information across them, with the aim of outperforming individual modeling. The key in MOGP is to construct a valid multi-output covariance function to transfer useful information across outputs. A pioneer and well-known model developed in the field of geostatistics is called linear model of coregionalization (LMC) (Journel and Huijbregts, 1978). This model constructs valid covariance functions by a linear combination of several Gaussian processes. Thereafter, various MOGPs have been developed and extended in the context of LMC Seeger et al. (2005), Bonilla et al. (2007), Hayashi et al. (2012), Osborne et al. (2012), Rakitsch et al. (2013), Dürichen et al. (2015), Hori et al. (2016). Another way to construct valid covariance functions is through process convolutions that convolve a base process, e.g., white Gaussian noise, with a smoothing kernel Ver Hoef and Barry (1998), Boyle and Frean (2004), Álvarez and Lawrence (2009), Álvarez and Lawrence (2011).1 The process convolutions can be regarded as a dynamic version of LMC Álvarez and Lawrence (2011), Álvarez et al. (2012).
In the multi-fidelity scenario, particularly, we attempt to use the inexpensive LF outputs to assist the modeling of the expensive HF output. Hence, compared to the typical MOGPs that share the information across the outputs, Kennedy and O’Hagan (2000) presented a Bayesian discrepancy-based MFM framework, which is an extension to the Co-Kriging model (Myers, 1982). In this framework, an auto-regressive model is proposed by expressing the HF output as the sum of the scaled LF outputs and an additive term that accounts for the discrepancy between the outputs, leading to not only the information sharing across outputs but also the asymmetric knowledge transfer from the LF outputs to the HF output. Later, Qian and Wu (2008) and Leen et al. (2012) provided an equivalent MFM framework in different views. Being a good multi-fidelity predictive model, Co-Kriging has been extended and improved, e.g., by reducing computational complexity Le Gratiet and Garnier (2014), Le Gratiet and Cannamela (2015), using space-dependent scaling factor (Perdikaris et al., 2017), and incorporating gradient information Han et al. (2013), Ulaganathan et al. (2015). Due to the remarkable performance, Co-Kriging has gained popularity in various fields, e.g., model inversion (Perdikaris and Karniadakis, 2016), uncertainty quantification Perdikaris et al. (2015), Kennedy and O’Hagan (2001), and multidisciplinary/robust/multi-objective optimization Forrester et al. (2007), Keane (2012), Han and Görtz (2012), Kontogiannis et al. (2017). For more information about Co-Kriging and MFM, one can refer to the recent reviews and comparison studies Fernández-Godino et al. (2016), Park et al. (2017), Toal (2015).
In the context of Co-Kriging, it is usually assumed that we can control the sampling process such that the HF and LF training points spread over the entire domain evenly by for example the nested sampling strategy (Qian, 2009) and the nearest neighbor sampling strategy (Le Gratiet and Garnier, 2014). The space-filling nested data structure, though being beneficial for Co-Kriging, cannot always be available in practice. In realistic scenarios, we need to handle diverse data structures. The diverse data structures here mainly refer to the diversity of HF sample distributions, while the inexpensive LF outputs are assumed to have sufficient training points that cover the entire domain. For example, as shown in Fig. 1,2 we have a set of uniformly distributed HF points, a set of randomly distributed HF points, and more challengingly, a set of partially distributed HF points clustered in a subregion. Besides, a practical example is that when using CFD solvers of different fidelities to simulate the flow around an aircraft, the inexpensive LF Euler simulation can be computed over the domain; while for saving computing time, the expensive HF Navier–Stokes simulation is only performed in flow regions with strong viscous effects. Hence, the diverse data structures, which contain different HF sample distributions, pose the demands of developing a particular multi-fidelity modeling approach that can effectively extract LF information to facilitate the HF modeling in different scenarios.
Therefore, this article presents a novel multi-fidelity GP model that is composed of a global trend term and a local residual term in order to tackle diverse data structures, e.g., full points that are available in the entire domain, or partial points that fill only some subregions. Particularly, in order to extract useful LF information effectively for facilitating the HF modeling, the local residual term contains a shared part and an output-specific part, the trade-off between which is dynamically determined by a data-driven weight parameter. The flexible model structure enables the approach to accomplish the multi-fidelity modeling well with diverse data structures.
The remainder of the article is organized as follows. Section 2 briefly introduces the single-output Gaussian process. Then, Section 3 presents the newly developed multi-fidelity Gaussian process in the MOGP framework. Thereafter, Section 4 comprehensively tests the new approach on two synthetic examples and two engineering examples with diverse characteristics and data structures. Finally, Section 5 offers some concluding remarks.
Section snippets
Single-output Gaussian process
Here we give a brief introduction to the single-output Gaussian process (SOGP). GP is a stochastic process wherein any finite subset of random variables follows a joint Gaussian distribution. As a non-parametric3 model, the GP interprets the target function where as a probability distribution
General multi-output Gaussian process
Different from the single-output case, here we consider the multi-output case that is frequently encountered in practice. In the multi-output scenario, we attempt to learn the mapping between the input space and the output space where is the number of outputs. The multi-output GP (MOGP) is to model outputs simultaneously by considering the output correlations, with the aim of outperforming individual modeling.
The construction of MOGP is similar to that of SOGP. As stated
Numerical experiments
This section validates the performance of the proposed MFGP modeling approach on four examples with diverse characteristics and data structures. Among them, we consider two synthetic examples (a 1D pedagogical example and a 2D Branin example) with , and then study two real-world engineering examples (a 6D Airfoil dataset and a 2D stochastic incompressible flow dataset) with and , respectively.
For the purpose of comparison, the testing involves four GP modeling approaches:
Conclusions
This article presents a novel multi-fidelity Gaussian process model for multi-fidelity regression problems with diverse data structures. Numerical results reveal that the proposed MFGP modeling approach has a promising performance in different scenarios. For different HF data structures, e.g., full HF points and partial HF points, the MFGP can effectively extract the LF information so as to enhance the modeling of the HF output. Future research efforts are needed to improve the modeling
Acknowledgments
This work was conducted within the Rolls-Royce@NTU Corporate Lab with support from the National Research Foundation (NRF) Singapore under the Corp Lab@University Scheme. It is also partially supported by the Data Science and Artificial Intelligence Research Center (DSAIR) and the School of Computer Science and Engineering at Nanyang Technological University .
References (48)
- et al.
Multi-output local Gaussian process regression: Applications to uncertainty quantification
J. Comput. Phys.
(2012) - et al.
Improving variable-fidelity surrogate modeling via gradient-enhanced kriging and a generalized hybrid bridge function
Aerosp. Sci. Technol.
(2013) - et al.
Multivariate versus univariate Kriging metamodels for multi-response simulation models
European J. Oper. Res.
(2014) - et al.
Constructing and fitting models for cokriging and multivariable spatial prediction
J. Statist. Plann. Inference
(1998) - et al.
On the condition number of covariance matrices in kriging, estimation, and simulation of random fields
Math. Geol.
(1994) - Álvarez, M., Lawrence, N.D., 2009. Sparse convolved Gaussian processes for multi-output regression. In: NIPS, pp....
- et al.
Computationally efficient convolved multiple output gaussian processes
J. Mach. Learn. Res.
(2011) - et al.
Kernels for vector-valued functions: A review
Found. Trends® Mach. Learn.
(2012) - Benamara, T., Breitkopf, P., Lepot, I., Sainvitu, C., 2016. Multi-fidelity extension to non-intrusive proper orthogonal...
- Bernstein, A., Burnaev, E., Chernova, S., Zhu, F., Qin, N., 2011. Comparison of three geometric parameterization...
Multitask Gaussian processes for multivariate physiological time-series analysis
IEEE Trans. Biomed. Eng.
Multi-fidelity optimization via surrogate modelling
Proc. R. Soc. A: Math. Phys. Eng. Sci.
Cases for the nugget in modeling computer experiments
Stat. Comput.
Hierarchical kriging model for variable-fidelity surrogate modeling
AIAA J.
Multi-task Gaussian process for imputing missing data in multi-trait and multi-environment trials
Theor. Appl. Genet.
Mining Geostatistics
Spectral/hp Element Methods for Computational Fluid Dynamics
Cokriging for robust design optimization
AIAA J.
Cited by (62)
A product performance rapid simulation approach driven by digital twin data: Part 1. For variable product structures
2024, Advanced Engineering InformaticsPiggybacking on past problem for faster optimization in aluminum electrolysis process design
2023, Engineering Applications of Artificial IntelligenceAdaptive data fusion framework for modeling of non-uniform aerodynamic data
2023, Chinese Journal of AeronauticsForecasting the eddy current loss of a large turbo generator using hybrid ensemble Gaussian process regression
2023, Engineering Applications of Artificial Intelligence