Monte Carlo evaluation of derivative-based global sensitivity measures

https://doi.org/10.1016/j.ress.2008.05.006Get rights and content

Abstract

A novel approach for evaluation of derivative-based global sensitivity measures (DGSM) is presented. It is compared with the Morris and the Sobol’ sensitivity indices methods. It is shown that there is a link between DGSM and Sobol’ sensitivity indices. DGSM are very easy to implement and evaluate numerically. The computational time required for numerical evaluation of DGSM is many orders of magnitude lower than that for estimation of the Sobol’ sensitivity indices. It is also lower than that for the Morris method. Efficiencies of Monte Carlo (MC) and quasi-Monte Carlo (QMC) sampling methods for calculation of DGSM are compared. It is shown that the superiority of QMC over MC depends on the problem's effective dimension, which can also be estimated using DGSM.

Introduction

Sensitivity analysis (SA) is widely used in engineering design. One of the widespread SA approaches is based on local methods. Local SA techniques are usually concerned with the computation of the derivative of the model response with respect to the model input parameters. The main disadvantage of these methods is that they do not account for interactions between variables and the local sensitivity coefficients are related to a fixed nominal point in the space of parameters.

Global SA offers a comprehensive approach to the model analysis. Unlike local SA, global SA methods evaluate the effect of a factor while all other factors are varied as well and thus they account for interactions between variables and do not depend on the choice of a nominal point. Reviews of different global SA methods can be found in [1], [2]. The method of global sensitivity indices suggested by Sobol’ in [3], and then further developed by Saltelli and Sobol’ in [4], Homma and Saltelli in [5] is one of the most efficient and popular global SA techniques. It belongs to the class of variance-based methods. These methods provide information on the importance of different subsets of input variables to the output variance. Variance-based methods generally require a large number of function evaluations to achieve reasonable convergence and can become impractical for large engineering problems. This is why a number of alternative SA techniques have been proposed recently. One of them is the screening method proposed by Morris [6]. It can be regarded as global as the final measure is obtained by averaging local measures (the elementary effects). This method is considerably cheaper than the variance-based methods in terms of computational time. The Morris method can be used for ranking and identifying unimportant variables. However, the Morris method has two main drawbacks. Firstly, it uses random sampling of points from the fixed grid (levels) for averaging elementary effects, which are calculated as finite differences with the increment delta comparable with the range of uncertainty. For this reason it cannot correctly account for the effects with characteristic dimensions much less than delta. Secondly, it lacks the ability of the Sobol’ method to provide information about main effects (contribution of individual variables to uncertainly) and it cannot distinguish between low-and high-order interactions.

In this paper, we present a new method, which we call derivative-based global sensitivity measures (DGSM). The method is based on averaging local derivatives using Monte Carlo (MC) or quasi-Monte Carlo (QMC) sampling methods. Our technique is much more accurate than the Morris method as the elementary effects are evaluated as strict local derivatives with small increments compared with the variable uncertainty ranges. Local derivatives are evaluated at randomly or quasi-randomly selected points in the whole range of uncertainty and not at the points from a fixed grid. We also introduce new sensitivity measures and demonstrate that there is a link between these measures and the Sobol’ sensitivity indices.

It is well known that for a sufficiently large number of sampled points N, QMC should always outperform MC [7]. However, for high-dimensional problems such a large number of points can be infeasible. Some numerical experiments demonstrated that the advantages of QMC integration can disappear for high-dimensional problems. At the same time there are high-dimensional problems for which QMC remains more efficient than MC. It was shown that the Sobol’ method can be used for the prediction of the QMC efficiency [8]. In this paper, we show that DGSM can also be used as quantitative measures of the QMC efficiency.

This paper is organised as follows: Section 2 introduces DGSM. Section 3 gives a brief description of Sobol’ global sensitivity indices. MC and QMC integration algorithms and low-discrepancy sequences are discussed in Section 4. This section also contains a description of the DGSM algorithm. Section 5 briefly describes the Morris method. Section 6 covers issues concerning the possible degradation of QMC efficiency in higher dimensions. It also introduces the notion of the effective dimension and presents the classification of functions based on Sobol’ global sensitivity indices. It is shown how this classification can be used for the prediction of the QMC efficiency. A comparison of the Sobol’ method, DGSM and the Morris methods is given in Section 7. Finally, conclusions are presented in Section 8.

Section snippets

Derivative-based global sensitivity measures

Consider a differentiable function f(x), where x={xi} is a vector of input variables defined in the unit hypercube Hn (0⩽xi⩽1, i=1, …, n). Local sensitivity measures are based on partial derivatives Ei(x*)=fxi.

The local sensitivity measure Ei(x*) depends on a nominal point x* and it changes with a change of x*. This deficiency can be overcome by averaging Ei(x*) over the parameter space Hn. Such a measure can be defined asM¯i=HnEidx.We also consider another measure, which is the standard

Sobol’ global sensitivity indices

The method of global sensitivity indices developed by Sobol’ is based on ANOVA decomposition [3]. Consider a square integrable function f(x) defined in the unit hypercube Hn. It can be expanded in the following form:f(x)=f0+ifi(xi)+i<jfij(xi,xj)++f12n(x1,x2,,xn).

This decomposition is unique if conditions01fi1...isdxik=0for1ks,are satisfied. Here 1⩽i1<⋯< isn. It follows from (9) and (10) thatf0=01f(x)dx.

The variances of the terms in the ANOVA decomposition add up to the total variance

Computational algorithms for calculation of derivative-based global sensitivity measures

Calculation of DGSM is based on the evaluation of integrals (2), (3), (4), (5), (6), (7), (8), which can be presented in the following generic form:I[f]=Hnf(x)dx.

It is assumed that function f(x) is integrable in the n-dimensional unit hypercube Hn.

Classical grid methods become inefficient in high dimensions because of the “curse of dimensionality” (exponential growth of the required integrand evaluations). MC methods do not depend on the dimensionality and are effective in high-dimensional

The Morris method

The Morris method is traditionally used as a screening method for problems with a high number of variables for which function evaluations can be CPU-time consuming. It is composed of individually randomized ‘one-factor-at-a-time’ experiments. Each input factor may assume a discrete number of values, called levels, which are chosen within the factor range of variation.

The sensitivity measures proposed in the original work of Morris [6] are based on what is called an elementary effect. It is

The effective dimension and classification of test functions

Generally, QMC is much more efficient than MC. However, some numerical experiments demonstrated that the advantages of QMC integration can disappear for high-dimensional problems [11]. There are also high-dimensional problems for which QMC outperforms MC [12]. Such behaviour can be explained by the problem's effective dimension. The notion of the “effective dimension” was introduced by Caflisch et al. [14].

Let |u| be a cardinality of a set u.

Definition 1

The effective dimension of f(x) in the superposition

Results

Results obtained by MC and QMC evaluations of DGSM are compared with the Morris method. DGSM are also compared with the Sobol’ sensitivity indices to see the analogy between the two types of measures. Seven different test functions were used for comparison (Table 3). Input variables for all functions are uniformly distributed in Hn with n=10 apart from the last function for which n=4.

The analytical values for the Sobol’ sensitivity indices and DGSM were calculated and compared with numerical

Conclusions

In this paper, we have considered derivative-based global sensitivity measures (DGSM). We have presented an efficient and general algorithm for evaluation of the DGSM based on MC and QMC integration methods. It has been proven that this approach can be much faster and more accurate than the Morris method. However, in many instances the Morris method still can be seen as a good compromise between accuracy and efficiency.

Different types of test functions have been used for testing and comparison.

Acknowledgements

The authors would like to thank I.M. Sobol’ and S.Tarantola for helpful discussions. We also thank A. Saltelli and two anonymous reviewers for their helpful comments. The financial support by the EPSRC Grant EP/D506743/1 is gratefully acknowledged. Author MRF also would like to acknowledge the financial support from the EU ERASysBio and the Spanish Ministry of Science and Innovation (SYSMO project “KOSMOBAC”, MEC GEN2006-27747-E/SYS).

References (16)

There are more references available in the full text version of this article.

Cited by (232)

  • An annotated timeline of sensitivity analysis

    2024, Environmental Modelling and Software
  • Screening: From tornado diagrams to effective dimensions

    2023, European Journal of Operational Research
View all citing articles on Scopus
View full text