Gradient-enhanced high dimensional model representation via Bayesian inference

Highlights

• HDMR surrogate model are developed based on Bayesian inference technique.

• An efficient method for HDMR model integrating gradient information is presented.

• Eight benchmark examples are used to validate the effectiveness of the method.

Abstract

Recently, gradient-enhanced surrogate models have drawn extensive attention for function approximation, in which the gradient information is utilized to improve the surrogate model accuracy. In this work, gradient-enhanced high dimensional model representation (HDMR) is established based on Bayesian inference technique. The proposed method first assigns Gaussian process prior for the model response function and its partial derivative functions (with respect to all the input variables). Then the auto-covariance functions and the cross-covariance functions of these random processes are established respectively by the HDMR basis functions. Finally, the posterior distribution of the response function is analytically obtained through Bayes theorem. The proposed method combines the sample information and gradient information in a seamless way to yield a highly accurate HDMR prediction model. We demonstrate our method via several examples, and the results all suggest that combining gradient information with sample information provides more accurate prediction results at reduced computational cost.

Introduction

Over the last decades, along with the rapid development of computer science and technique, a variety of complex computational models have been developed for simulating and predicting the behavior of systems in nearly all fields of engineering and science. In the meanwhile, surrogate model is becoming more and more popular to meet the growing engineering demand in computational power [1]. For decades, various types of surrogate model techniques have been continuously developed, such as Kriging/Gaussian Process (GP) [2], [3], [4], [5], [6] and its gradient-enhanced versions [7], support vector regression (SVR) [8], [9], [10], [11], [12]and its gradient-enhanced variant [13], [14], [15], radial basis functions (RBF) [16], [17], [18], polynomial chaos expansion (PCE) [13], [19], [20], [21], [22], [23], artificial neural networks (ANN) [24], [25], [26] and high dimensional model representation (HDMR) [27], [28], [29], [30], [31], [32], [33], [34], [35], among which the HDMR has attracted significant interest due to its ability to handle complex high dimensional models.

HDMR is developed as a set of quantitative model assessment and analysis tools for capturing complex high-dimensional input–output system behavior [31], [32], [36]. Its theoretical foundation lies in that only low-order correlations amongst the input variables have a significant effect on the model outputs, thus the HDMR permits expressing a multidimensional function as a sum of many low dimensional component functions to reflect the independent and cooperative contribution. In practical applications, two types of HDMR expansions are developed in literature. One decomposes models output into components defined on hyper-planes based on a fixed cut point or anchor point in the parameter space, thus this method is known as Cut-HDMR. The other method conforms to the analysis of variance (ANOVA) decomposition widely used in uncertainty quantification and sensitivity analysis, and hence is called ANOVA-HDMR.

To construct the HDMR surrogate model, various methods are developed to evaluate the low-order component terms of the truncated dimension-wise decompositions. For Cut-HDMR, the usual practice is to set up a look-up table of the component functions values evaluated at the selected cut points or anchor points in the parameter space, and the Lagrange or linear interpolation method is utilized to estimate the component functions. Rabitz et al. [37] used equidistantly distributed sample points along each axis of the input parameter to develop Cut-HDMR. Liu et al. [27] suggested using non-uniform optimal nodes to mitigate the Runge’s phenomenon. The optimal nodes are selected as the nodes of Legendre–Gauss, Chebyshev—Gauss and Clenshaw–Curtis quadratures. ANOVA-HDMR, also known as random sampling (RS)-HDMR, expands the component functions in terms of analytical basis functions, and the Monte Carlo simulations or other technique is utilized to estimate the basis function coefficients. Li et al. [36] suggested using orthogonal polynomials, cubic B-splines and polynomial to approximate the ANOVA-HDMR component functions. Luo et al. [34] used the reproducing kernel technique to estimate the arbitrary order ANOVA-HDMR component functions. Lambert et al. [38] utilized the group method of data handling (GMDH) method to construct the ANOVA-HDMR surrogate model based on Legendre polynomial basis functions.

This work aims at developing the ANOVA-HDMR surrogate model integrating with gradient information based on Bayesian inference technique, namely, gradient-enhanced HDMR (GE-HDMR). Firstly, the response function of an underlying model is approximated by the HDMR component functions with unknown coefficients. Then we assign the GP prior for the HDMR model and its partial derivative function. The auto-covariance functions and the cross-covariance functions of these GP are defined by the HDMR component functions respectively. Given a set of samples (response information and gradient information), the posterior distribution of the model response can be computed by means of Bayes theorem, and the GE-HDMR surrogate model is analytically derived by its mean function. The proposed method combines the sample information and gradient information to approximate the response function, thus it provides much more accurate prediction result compared to classic HDMR model with no gradient information. Several examples are used to validate the accuracy and efficiency of the presented method, results demonstrate that the developed GE-HDMR surrogate model is much more efficient and accurate than the classic HDMR model.

The rest of this paper is organized as follows: Section 2 reviews the basic theories of classic HDMR. The construction of the GE-HDMR surrogate model is introduced in detail in Section 3. In Section 4, several tested examples are used to illustrate the performance of the proposed GE-HDMR method. Finally, some conclusions are drawn in Section 5.