Global sensitivity analysis for mathematical models comparison

Abstract

The choice of an adequate mathematical model is a key step in solving problems in many different fields. When more than one model is available to represent a given phenomenon, a poor choice might result in loss of precision and efficiency. Well-known strategies for comparing mathematical models can be found in many previous works, but seldom regarding several models with uncertain parameters at once. In this work, we present a novel approach for measuring the similarity among any given number of mathematical models, so as to support decision making regarding model selection. The strategy consists in defining a new general model composed of all candidate models and a uniformly distributed random variable, whose sampling selects the candidate model employed to evaluate the response. Global Sensitivity Analysis (GSA) is then performed to measure the sensitivity of the response with respect to this random variable. The result indicates the level of discrepancy among the mathematical models in the stochastic context. We also demonstrate that the proposed approach is related to the Root Mean Square (RMS) error when only two models are compared. The main advantages of the proposed approach are: (i) the problem is cast in the sound framework of GSA, (ii) the approach also quantifies if the discrepancy among the mathematical models is significant in comparison to uncertainties/randomness of the parameters, an analysis that is not possible with RMS error alone. Numerical examples of different disciplines and degrees of complexity are presented, showing the kind of insight we can get from the proposed approach.

Appendix A relation to mean square error

The total index with respect to model choice can be written as

$$\begin{aligned} \begin{aligned} S_{TW}&= 1 - \frac{\mathbb {V}_X\left[ \mathbb {E}_W\left[ Y \vert {\textbf{X}}_{\sim W}\right] \right] }{\mathbb {V}\left[ Y\right] }\\&= \frac{\mathbb {E}_X\left[ \mathbb {V}_W\left[ Y \vert {\textbf{X}}_{\sim W}\right] \right] }{\mathbb {V}\left[ Y\right] }.\\ \end{aligned} \end{aligned}$$

(A1)

Let us define the auxiliary random variable

$$\begin{aligned} Z(X) = \mathbb {V}_W\left[ Y \vert {\textbf{X}}_{\sim W}\right] . \end{aligned}$$

(A2)

If only two mathematical models $f_1$, $f_2$ are considered, we then have

$$\begin{aligned} Z(X) = \frac{1}{2} \left( f_1(X) - \mathbb {E}_W\left[ Y \vert {\textbf{X}}_{\sim W}\right] \right) ^2 + \frac{1}{2} \left( f_2(X) - \mathbb {E}_W\left[ Y \vert {\textbf{X}}_{\sim W}\right] \right) ^2. \end{aligned}$$

(A3)

Besides

$$\begin{aligned} \mathbb {E}_W\left[ Y\vert {\textbf{X}}_{\sim W}\right] = \frac{1}{2} \left( f_1(X) + f_2(X) \right) . \end{aligned}$$

(A4)

Thus

$$\begin{aligned} \begin{aligned} f_1(X) - \mathbb {E}_W\left[ Y\vert {\textbf{X}}_{\sim W}\right]&= f_1(X) - \frac{1}{2} \left( f_1(X) + f_2(X) \right) \\&= \frac{1}{2} \left( f_1(X) - f_2(X) \right) \end{aligned} \end{aligned}$$

(A5)

and

$$\begin{aligned} \begin{aligned} \left( f_1(X) - \mathbb {E}_W\left[ Y\vert {\textbf{X}}_{\sim W}\right] \right) ^2&= \left( f_2(X) - \mathbb {E}_W\left[ Y\vert {\textbf{X}}_{\sim W}\right] \right) ^2\\&= \frac{1}{4} \left( f_2(X) - f_1(X) \right) ^2. \end{aligned} \end{aligned}$$

(A6)

Substitution of Eq. (A6) into Eq. (A3) gives

$$\begin{aligned} Z(X) = \frac{1}{4} \left( f_2(X) - f_1(X) \right) ^2. \end{aligned}$$

(A7)

Consequently

$$\begin{aligned} \begin{aligned} \mathbb {E}_X\left[ \mathbb {V}_W\left[ Y\vert {\textbf{X}}_{\sim W}\right] \right]&= \mathbb {E}_X \left[ Z(X)\right] \\&= \frac{1}{4} \mathbb {E}_X \left[ \left( f_2(X) - f_1(X) \right) ^2\right] . \end{aligned} \end{aligned}$$

(A8)

The total index with respect to model choice can then be written as

$$\begin{aligned} S_{TW} = \frac{1}{4} \frac{\mathbb {E}_X \left[ \left( f_2(X) - f_1(X) \right) ^2\right] }{\mathbb {V}\left[ Y \right] } \end{aligned}$$

(A9)

However, we observe that the Root Mean Square (RMS) Error is defined as

$$\begin{aligned} E_{RMS} = \sqrt{\mathbb {E}_X \left[ \left( f_2(X) - f_1(X) \right) ^2\right] }. \end{aligned}$$

(A10)

For this reason we can write

$$\begin{aligned} S_{TW} = \frac{1}{4} \frac{E_{RMS}^2}{\mathbb {V}\left[ Y \right] } \end{aligned}$$

(A11)

or, alternatively,

$$\begin{aligned} E_{RMS} = 2 \sqrt{ S_{TW} \mathbb {V}\left[ Y \right] }. \end{aligned}$$

(A12)

This demonstrates Eq. (8).

Also note that

$$\begin{aligned} \mathbb {V}\left[ Y \right] = \mathbb {E}_X\left[ \mathbb {V}_W\left[ Y\vert {\textbf{X}}_{\sim W}\right] \right] + \mathbb {V}_X\left[ \mathbb {E}_W\left[ Y\vert {\textbf{X}}_{\sim W}\right] \right] \end{aligned}$$

(A13)

and

$$\begin{aligned} \mathbb {V}_X\left[ \mathbb {E}_W\left[ Y\vert {\textbf{X}}_{\sim W}\right] \right] = \mathbb {V}_X\left[ \frac{1}{2} \left( f_1(X) + f_2(X) \right) \right] . \end{aligned}$$

(A14)

Thus

$$\begin{aligned} \mathbb {V}\left[ Y \right] = \frac{1}{4} \mathbb {E}_X \left[ \left( f_2(X) - f_1(X) \right) ^2\right] + \mathbb {V}_X\left[ \frac{1}{2} \left( f_1(X) + f_2(X) \right) \right] . \end{aligned}$$

(A15)

It is then possible to write

$$\begin{aligned} \mathbb {V}\left[ Y \right] = \frac{1}{4} E_{RMS}^2 + \mathbb {V}_X\left[ m(X) \right] , \end{aligned}$$

(A16)

where

$$\begin{aligned} m(X) = \frac{1}{2} \left( f_1(X) + f_2(X) \right) \end{aligned}$$

(A17)

is the mean model, i.e. the point-wise mean between models $f_1$ and $f_2$. From these results we can also write

$$\begin{aligned} S_{TW} = \frac{E_{RMS}^2}{E_{RMS}^2 + 4 \mathbb {V}_X\left[ m(X) \right] }, \end{aligned}$$

(A18)

that demonstrates Eq. (9).