An efficient dimension reduction for the Gaussian process emulation of two nested codes with functional outputs

Abstract

In this paper, we first propose an efficient method for the dimension reduction of the functional input of a code with functional output. It is based on the approximation of the output by a model which is linear with respect to the functional input. This approximation has a sparse structure, whose parameters can be accurately estimated from a small set of observations of the code. The Gaussian predictor based on this projection basis is significantly more accurate than the one based on a projection obtained with Partial Least Squares. Secondly, the surrogate modeling of two nested codes with functional outputs is considered. In such a case, the functional output of the first code is one of the inputs of the second code. The Gaussian process regression of the second code is performed using the proposed dimension reduction. A Gaussian predictor of the nested code is obtained by composing the predictors of the two codes and linearizing this composition. Moreover, two sequential design criteria are proposed. Since we aim at performing a sensitivity analysis, these criteria are based on a minimization of the prediction variance. Moreover, one of the criteria enables to choose, if it is possible, which of the two codes to run. Thus, the computational budget is optimally allocated between the two codes and the prediction error is substantially reduced.

Appendix

1.1 Proof of Proposition 1

We aim at finding a m-rank matrix $\varvec{Z}^*$ such that:

$$\begin{aligned} \varvec{Z}^*=\underset{\scriptstyle \varvec{Z} \in \mathcal {M}_{N_t\times N_t}}{{\text {argmin}}}\ \sum \limits _{i=1}^{N_x} \left\| \varvec{Ax}_t^{(i)}-\varvec{AZx}_t^{(i)} \right\| ^2, \end{aligned}$$

where $\varvec{Z}$ is a m-rank matrix and $\varvec{x}_t^{(i)}$ denotes the i-th observation of the input.

The previous equation can be rewritten:

$$\begin{aligned} \varvec{Z}^*= & {} \underset{\scriptstyle \varvec{Z} \in \mathcal {M}_{N_t\times N_t}}{{\text {argmin}}}\ \sum \limits _{i=1}^{N_x} \left\| \varvec{Ax}_t^{(i)}-\varvec{AZx}_t^{(i)} \right\| ^2 \\= & {} \underset{\scriptstyle \varvec{Z} \in \mathcal {M}_{N_t\times N_t}}{{\text {argmin}}}\ \text {Tr}\left( \left( \varvec{AX}_t-\varvec{AZX}_t \right) ^T\left( \varvec{AX}_t-\varvec{AZX}_t \right) \right) \\= & {} \underset{\scriptstyle \varvec{Z} \in \mathcal {M}_{N_t\times N_t}}{{\text {argmin}}}\ \text {Tr}\left( \left( \varvec{AZ}^c\varvec{X}_t \right) ^T\left( \varvec{AZ}^c\varvec{X}_t \right) \right) \\= & {} \underset{\scriptstyle \varvec{Z} \in \mathcal {M}_{N_t\times N_t}}{{\text {argmin}}}\ \text {Tr}\left( \varvec{X}_t^T\left( \varvec{Z}^c \right) ^T\varvec{A}^T \varvec{AZ}^c\varvec{X}_t \right) \\= & {} \underset{\scriptstyle \varvec{Z} \in \mathcal {M}_{N_t\times N_t}}{{\text {argmin}}}\ \text {Tr}\left( \left( \varvec{Z}^c \right) ^T\varvec{A}^T \varvec{AZ}^c\varvec{X}_t\varvec{X}_t^T \right) \\= & {} \underset{\scriptstyle \varvec{Z} \in \mathcal {M}_{N_t\times N_t}}{{\text {argmin}}}\ \text {Tr}\left( \left( \varvec{X}_t\varvec{X}_t^T \right) ^{\frac{1}{2}}\left( \varvec{Z}^c \right) ^T\varvec{A}^T \varvec{AZ}^c\left( \varvec{X}_t\varvec{X}_t^T \right) ^{\frac{1}{2}} \right) \\= & {} \underset{\scriptstyle \varvec{Z} \in \mathcal {M}_{N_t\times N_t}}{{\text {argmin}}}\ \text {Tr}\left( \left( \varvec{X}_t\varvec{X}_t^T \right) ^{\frac{1}{2}}\left( \varvec{Z}^c \right) ^T\left( \varvec{A}^T\varvec{A} \right) ^{\frac{1}{2}} \left( \varvec{A}^T\varvec{A} \right) ^{\frac{1}{2}} \varvec{Z}^c\left( \varvec{X}_t\varvec{X}_t^T \right) ^{\frac{1}{2}} \right) \\= & {} \underset{\scriptstyle \varvec{Z} \in \mathcal {M}_{N_t\times N_t}}{{\text {argmin}}}\ \text {Tr}\left( \left( \left( \varvec{A}^T\varvec{A} \right) ^{\frac{1}{2}} \varvec{Z}^c\left( \varvec{X}_t\varvec{X}_t^T \right) ^{\frac{1}{2}} \right) ^T \left( \varvec{A}^T\varvec{A} \right) ^{\frac{1}{2}} \varvec{Z}^c\left( \varvec{X}_t\varvec{X}_t^T \right) ^{\frac{1}{2}} \right) \\= & {} \underset{\scriptstyle \varvec{Z} \in \mathcal {M}_{N_t\times N_t}}{{\text {argmin}}}\ \left\| \left( \varvec{A}^T\varvec{A} \right) ^{\frac{1}{2}} \varvec{Z}^c\left( \varvec{X}_t\varvec{X}_t^T \right) ^{\frac{1}{2}} \right\| ^2_F \\= & {} \underset{\scriptstyle \varvec{Z} \in \mathcal {M}_{N_t\times N_t}}{{\text {argmin}}}\ \left\| \left( \varvec{A}^T\varvec{A} \right) ^{\frac{1}{2}} \left( \varvec{X}_t\varvec{X}_t^T \right) ^{\frac{1}{2}} - \left( \varvec{A}^T\varvec{A} \right) ^{\frac{1}{2}} \varvec{Z}\left( \varvec{X}_t\varvec{X}_t^T \right) ^{\frac{1}{2}} \right\| ^2_F. \end{aligned}$$

In Eckart and Young (1936), it is shown that the matrix $\varvec{Q}_m$ of rank m which minimizes $\left\| \left( \varvec{A}^T\varvec{A} \right) ^{\frac{1}{2}} \left( \varvec{X}_t\varvec{X}_t^T \right) ^{\frac{1}{2}} - \varvec{Q}_m \right\| ^2_F$ is given by the m first singular-values of the singular-value decomposition of $\left( \varvec{A}^T\varvec{A} \right) ^{\frac{1}{2}} \left( \varvec{X}_t\varvec{X}_t^T \right) ^{\frac{1}{2}}$. If we denote by $\varvec{U D V}^T$ the singular-value decomposition of $\left( \varvec{A}^T\varvec{A} \right) ^{\frac{1}{2}}\left( \varvec{X}_t\varvec{X}_t^T \right) ^{\frac{1}{2}}$, with $\varvec{U}$ and $\varvec{V}$ gathering the singular vectors associated with the singular values sorted in ascending order, then we have $\varvec{Q}_m=\varvec{U}_m \varvec{D}_m \varvec{V}_m^T$ where $\varvec{U}_m$ and $\varvec{V}_m$ gather the m first columns of $\varvec{U}$ and $\varvec{V}$ and $\varvec{D}_m$ contains the m first lines and columns of $\varvec{D}$. It can therefore be inferred that:

$$\begin{aligned} \left( \varvec{A}^T\varvec{A} \right) ^{\frac{1}{2}} \varvec{Z}^*\left( \varvec{X}_t\varvec{X}_t^T \right) ^{\frac{1}{2}}=\varvec{U}_m \varvec{D}_m \varvec{V}_m^T. \end{aligned}$$

Hence, we have:

$$\begin{aligned} \varvec{Z}^*=\left( \varvec{A}^T\varvec{A} \right) ^{-\frac{1}{2}}\varvec{U}_m \varvec{D}_m \varvec{V}_m^T\left( \varvec{X}_t\varvec{X}_t^T \right) ^{-\frac{1}{2}}. \end{aligned}$$

1.2 Proof of Lemma 1

1.2.1 First equation

From Eq. (35), it can be inferred that:

$$\begin{aligned}&\mathbb {E}\left[ \varvec{M}_i \vert \lbrace \varvec{Y}_{i}^{\text {obs}},\varvec{Y}_i\left( \varvec{\bar{x}}_{i}^* \right) = \varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) \rbrace , \varvec{R}_{t_i}, C_i \right] = \varvec{Y}_i^{\text {obs,new}}\left( \varvec{R}_i^{\text {obs,new}} \right) ^{-1}\left( \varvec{H}_i^{\text {obs,new}} \right) ^T \\&\qquad \left( \varvec{H}_i^{\text {obs,new}}\left( \varvec{R}_i^{\text {obs,new}} \right) ^{-1}\left( \varvec{H}_i^{\text {obs,new}} \right) ^T \right) ^{-1}, \end{aligned}$$

where

$$\begin{aligned} \begin{array}{rl} \varvec{Y}_i^{\text {obs,new}} &{}=\left( \varvec{Y}_i^\text {obs}\ ,\ \varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) \right) , \\ \varvec{H}_i^{\text {obs,new}} &{}=\left( \varvec{H}_i^\text {obs}\ ,\ \varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) , \end{array} \end{aligned}$$

(73)

and

$$\begin{aligned} \varvec{R}_i^{\text {obs,new}}= \left( \begin{array}{cc} \varvec{R}_i^{\text {obs,new}} &{} C\left( \overline{\varvec{X}}_{i}^{\text {obs}},\varvec{\bar{x}}_{i}^* \right) \\ C\left( \varvec{\bar{x}}_{i}^*,\overline{\varvec{X}}_{i}^{\text {obs}} \right) &{} 1 \end{array} \right) . \end{aligned}$$

Using the Schur complement formulas, one gets:

$$\begin{aligned} \left( \varvec{R}_i^{\text {obs,new}} \right) ^{-1}= \left( \begin{array}{cc} \varvec{Q}_{11}&{} \varvec{Q}_{12} \\ \varvec{Q}_{21} &{} \varvec{Q}_{22} \end{array} \right) , \end{aligned}$$

(74)

where

$$\begin{aligned} \begin{array}{rl} \varvec{Q}_{11} &{}= \left( \varvec{R}_i^\text {obs} \right) ^{-1}+\varvec{Q}_{22}\left( \varvec{R}_i^\text {obs} \right) ^{-1}C\left( \overline{\varvec{X}}_{i}^{\text {obs}},\varvec{\bar{x}}_{i}^* \right) C\left( \varvec{\bar{x}}_{i}^*,\overline{\varvec{X}}_{i}^{\text {obs}} \right) \left( \varvec{R}_i^\text {obs} \right) ^{-1}\\ &{} =\left( \varvec{R}_i^\text {obs} \right) ^{-1}+\varvec{Q}_{22}\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T, \\ \varvec{Q}_{12} &{} =-\varvec{Q}_{22}\left( \varvec{R}_i^\text {obs} \right) ^{-1}C\left( \overline{\varvec{X}}_{i}^{\text {obs}},\varvec{\bar{x}}_{i}^* \right) =-\varvec{Q}_{22}\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) , \\ \varvec{Q}_{21} &{} =-\varvec{Q}_{22}C\left( \varvec{\bar{x}}_{i}^*,\overline{\varvec{X}}_{i}^{\text {obs}} \right) \left( \varvec{R}_i^\text {obs} \right) ^{-1}=-\varvec{Q}_{22}\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T, \end{array} \end{aligned}$$

(75)

$\varvec{Q}_{22}\in \mathbb {R}$, and:

$$\begin{aligned} \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) =\left( \varvec{R}_i^\text {obs} \right) ^{-1}C\left( \overline{\varvec{X}}_{i}^{\text {obs}},\varvec{\bar{x}}_{i}^* \right) . \end{aligned}$$

(76)

From the previous equations, one has:

$$\begin{aligned}&\varvec{H}_i^{\text {obs,new}}\left( \varvec{R}_i^{\text {obs,new}} \right) ^{-1}\left( \varvec{H}_i^{\text {obs,new}} \right) ^T\nonumber \\&\quad = \varvec{H}_i^\text {obs}\varvec{Q}_{11}\left( \varvec{H}_i^\text {obs} \right) ^T+ \varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{Q}_{21}\left( \varvec{H}_i^\text {obs} \right) ^T +\varvec{H}_i^\text {obs}\varvec{Q}_{12}\varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T\nonumber \\&\qquad +\, \varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{Q}_{22}\varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T\nonumber \\&\quad =\varvec{H}_i^\text {obs}\left( \varvec{R}_i^\text {obs} \right) ^{-1}\left( \varvec{H}_i^\text {obs} \right) ^T +\varvec{Q}_{22}\left( \varvec{H}_i^\text {obs}\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T\left( \varvec{H}_i^\text {obs} \right) ^T \right. \nonumber \\&\qquad \left. -\varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T\left( \varvec{H}_i^\text {obs} \right) ^T \right. \nonumber \\&\qquad \left. -\varvec{H}_i^\text {obs}\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T+ \varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T\right) \nonumber \\&\quad =\varvec{H}_i^\text {obs}\left( \varvec{R}_i^\text {obs} \right) ^{-1}\left( \varvec{H}_i^\text {obs} \right) ^T +\varvec{Q}_{22}\left( \varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) \right. \nonumber \\&\left. \qquad -\varvec{H}_i^\text {obs}\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) \left( \varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) -\varvec{H}_i^\text {obs}\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) ^T\nonumber \\&\quad =\varvec{H}_i^\text {obs}\left( \varvec{R}_i^\text {obs} \right) ^{-1}\left( \varvec{H}_i^\text {obs} \right) ^T +\varvec{Q}_{22}\varvec{u}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{u}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T, \end{aligned}$$

(77)

where $u_{i}$ is defined by Eq. (34). In the same way, one has:

$$\begin{aligned}&\varvec{Y}_i^{\text {obs,new}}\left( \varvec{R}_i^{\text {obs,new}} \right) ^{-1}\left( \varvec{H}_i^{\text {obs,new}} \right) ^T\nonumber \\&\quad = \varvec{Y}_i^\text {obs}\varvec{Q}_{11}\left( \varvec{H}_i^\text {obs} \right) ^T+ \varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{Q}_{21}\left( \varvec{H}_i^\text {obs} \right) ^T +\varvec{Y}_i^\text {obs}\varvec{Q}_{12}\varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T\nonumber \\&\qquad + \varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{Q}_{22}\varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T \nonumber \\&\quad =\varvec{Y}_i^\text {obs}\left( \varvec{R}_i^\text {obs} \right) ^{-1}\left( \varvec{H}_i^\text {obs} \right) ^T +\varvec{Q}_{22}\left( \varvec{Y}_i^\text {obs}\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T\left( \varvec{H}_i^\text {obs} \right) ^T\right. \nonumber \\&\qquad \left. - \varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T\left( \varvec{H}_i^\text {obs} \right) ^T \right. \nonumber \\&\qquad \left. -\varvec{Y}_i^\text {obs}\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T+ \varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T\right) \nonumber \\&\quad =\varvec{Y}_i^\text {obs}\left( \varvec{R}_i^\text {obs} \right) ^{-1}\left( \varvec{H}_i^\text {obs} \right) ^T \nonumber \\&\qquad +\varvec{Q}_{22}\left( \varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) -\varvec{Y}_i^\text {obs}\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) \left( \varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) -\left( \varvec{H}_i^\text {obs} \right) \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) ^T \nonumber \\&\quad =\varvec{Y}_i^\text {obs}\left( \varvec{R}_i^\text {obs} \right) ^{-1}\left( \varvec{H}_i^\text {obs} \right) ^T +\varvec{Q}_{22}\left( \varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) -\varvec{Y}_i^\text {obs}\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) \varvec{u}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T. \end{aligned}$$

From Eqs. (34) and (76), it can be inferred that:

$$\begin{aligned} \varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) - \varvec{Y}_i^\text {obs}\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) =\widehat{\varvec{M}}_i\varvec{u}_i\left( \varvec{\bar{x}}_{i}^* \right) , \end{aligned}$$

where $\widehat{\varvec{M}}_i=\mathbb {E}\left[ \varvec{M}_i \vert \varvec{Y}_{i}^{\text {obs}}, C_i \right]$ is defined by Eq. (35).

Therefore, one has:

$$\begin{aligned}&\varvec{Y}_i^{\text {obs,new}}\left( \varvec{R}_i^{\text {obs,new}} \right) ^{-1}\left( \varvec{H}_i^{\text {obs,new}} \right) ^T\\&\quad =\varvec{Y}_i^\text {obs}\left( \varvec{R}_i^\text {obs} \right) ^{-1}\left( \varvec{H}_i^\text {obs} \right) ^T +\varvec{Q}_{22}\widehat{\varvec{M}}_i\varvec{u}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{u}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T. \end{aligned}$$

From Eqs. (35) and (77) it can be inferred that:

$$\begin{aligned}&\varvec{Y}_i^{\text {obs,new}}\left( \varvec{R}_i^{\text {obs,new}} \right) ^{-1}\left( \varvec{H}_i^{\text {obs,new}} \right) ^T\nonumber \\&\quad =\widehat{\varvec{M}}_i\varvec{H}_i^\text {obs}\left( \varvec{R}_i^\text {obs} \right) ^{-1}\left( \varvec{H}_i^\text {obs} \right) ^T +\varvec{Q}_{22}\widehat{\varvec{M}}_i\varvec{u}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{u}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T\nonumber \\&\quad =\widehat{\varvec{M}}_i\left( \varvec{H}_i^\text {obs}\left( \varvec{R}_i^\text {obs} \right) ^{-1}\left( \varvec{H}_i^\text {obs} \right) ^T +\varvec{Q}_{22}\varvec{u}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{u}_i\left( \varvec{\bar{x}}_{i}^* \right) ^T \right) \nonumber \\&\quad =\widehat{\varvec{M}}_i\varvec{H}_i^{\text {obs,new}}\left( \varvec{R}_i^{\text {obs,new}} \right) ^{-1}\left( \varvec{H}_i^{\text {obs,new}} \right) ^T. \end{aligned}$$

Therefore, one has:

$$\begin{aligned} \varvec{Y}_i^{\text {obs,new}}\left( \varvec{R}_i^{\text {obs,new}} \right) ^{-1}\left( \varvec{H}_i^{\text {obs,new}} \right) ^T\left( \varvec{H}_i^{\text {obs,new}}\left( \varvec{R}_i^{\text {obs,new}} \right) ^{-1}\left( \varvec{H}_i^{\text {obs,new}} \right) ^T \right) ^{-1} =\widehat{\varvec{M}}_i. \end{aligned}$$

Thus, one can conclude from the previous equation and Eq.  (35) that:

$$\begin{aligned} \mathbb {E}\left[ \varvec{M}_i \vert \lbrace \varvec{Y}_{i}^{\text {obs}},\varvec{Y}_i\left( \varvec{\bar{x}}_{i}^* \right) = \varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) \rbrace , \varvec{R}_{t_i}, C_i \right] =\mathbb {E}\left[ \varvec{M}_i \vert \varvec{Y}_{i}^{\text {obs}}, \varvec{R}_{t_i}, C_i \right] . \end{aligned}$$

1.2.2 Second equation

From Eq. (36) it can be inferred that:

$$\begin{aligned}&\varvec{R}_{t_i}\left( \varvec{Y}_i^\text {obs},\varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) \right) = \dfrac{1}{n_i+1} \left( \varvec{Y}_i^{\text {obs,new}}-\widehat{\varvec{M}}_i\varvec{H}_i^{\text {obs,new}} \right) \nonumber \\&\quad \left( \varvec{R}_i^{\text {obs,new}} \right) ^{-1} \left( \varvec{Y}_i^{\text {obs,new}}-\widehat{\varvec{M}}_i\varvec{H}_i^{\text {obs,new}} \right) ^T. \end{aligned}$$

(78)

From Eq. (73), one has:

$$\begin{aligned} \varvec{Y}_i^{\text {obs,new}}-\widehat{\varvec{M}}_i\varvec{H}_i^{\text {obs,new}}=\left( \varvec{Y}_i^\text {obs} -\widehat{\varvec{M}}_i\varvec{H}_i^\text {obs} \ ,\ \varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) -\widehat{\varvec{M}}_i\varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) . \end{aligned}$$

(79)

From Eq. (34), one gets:

$$\begin{aligned} \varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) -\widehat{\varvec{M}}_i\varvec{h}_i\left( \varvec{\bar{x}}_{i}^* \right) =\left( \varvec{Y}_i^\text {obs} -\widehat{\varvec{M}}_i\varvec{H}_i^\text {obs} \right) \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) , \end{aligned}$$

where $\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right)$ is defined by Eq. (76).

Therefore, Eq. (79) becomes:

$$\begin{aligned} \varvec{Y}_i^{\text {obs,new}}-\widehat{\varvec{M}}_i\varvec{H}_i^{\text {obs,new}}= \left( \varvec{Y}_i^\text {obs} -\widehat{\varvec{M}}_i\varvec{H}_i^\text {obs} \right) \left( \varvec{I}_{n_i}\ ,\ \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) . \end{aligned}$$

Eq. (78) can thus be rewritten:

$$\begin{aligned} \varvec{R}_{t_i}\left( \varvec{Y}_i^\text {obs},\varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) \right)= & {} \dfrac{1}{n_i+1} \left( \varvec{Y}_i^\text {obs} -\widehat{\varvec{M}}_i\varvec{H}_i^\text {obs} \right) \left( \varvec{I}_{n_i}\ ,\ \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) \left( \varvec{R}_i^{\text {obs,new}} \right) ^{-1} \nonumber \\&\times \, \left( \varvec{I}_{n_i}\ ,\ \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) ^T \left( \varvec{Y}_i^\text {obs} -\widehat{\varvec{M}}_i\varvec{H}_i^\text {obs} \right) ^T. \end{aligned}$$

(80)

Besides, one has from Eqs. (74) and (75):

$$\begin{aligned} \begin{array}{rl} \left( \varvec{I}_{n_i}\ ,\ \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) \left( \varvec{R}_i^{\text {obs,new}} \right) ^{-1}= &{} \left( \varvec{I}_{n_i} \ ,\ \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) \left( \begin{array}{cc} \varvec{Q}_{11}&{} \varvec{Q}_{12} \\ \varvec{Q}_{21} &{} \varvec{Q}_{22} \end{array} \right) \\ = &{} \left( \varvec{Q}_{11}+\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{Q}_{21} \ ,\ \varvec{Q}_{12}+\varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \varvec{Q}_{22} \right) \\ = &{} \left( \left( \varvec{R}_i^\text {obs} \right) ^{-1} \ ,\ \varvec{0}_{n_i} \right) . \end{array} \end{aligned}$$

One can infer that:

$$\begin{aligned} \left( \varvec{I}_{n_i} \ ,\ \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) \left( \begin{array}{cc} \varvec{Q}_{11}&{} \varvec{Q}_{12} \\ \varvec{Q}_{21} &{} \varvec{Q}_{22} \end{array}, \right) \left( \begin{array}{c} \varvec{I}_{n_i} \\ \left( \varvec{v}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) ^T \end{array} \right) =\left( \varvec{R}_i^\text {obs} \right) ^{-1}. \end{aligned}$$

One can conclude from Eq. (80), (36) and the previous equation, that:

$$\begin{aligned} \varvec{R}_{t_i}\left( \varvec{Y}_i^\text {obs},\varvec{\mu }^c_i\left( \varvec{\bar{x}}_{i}^* \right) \right) =\dfrac{n_i}{n_i+1}\varvec{R}_{t_i}\left( \varvec{Y}_i^\text {obs} \right) . \end{aligned}$$

1.3 Proof of Proposition 2

From Eq. (39) it can be inferred that the criterion can be rewritten:

$$\begin{aligned} \varvec{\bar{x}}_{i}^{new}=\underset{\scriptstyle \varvec{\bar{x}}_{i}^* \in \overline{\mathbb {X}}_i}{{\text {argmin}}} \displaystyle \int _{\overline{\mathbb {X}}_i} \text {Tr}\left( \varvec{R}_{t_i}\left( \varvec{Y}_i^\text {obs},\varvec{y}_i\left( \varvec{\bar{x}}_{i}^* \right) \right) \right) v_i\left( \varvec{\bar{x}}_{i}\vert \varvec{\overline{X}}_{i}^{\text {obs}},\varvec{\bar{x}}_{i}^* \right) d\mu _{\overline{\mathbb {X}}_i}\left( \varvec{\bar{x}}_{i} \right) . \end{aligned}$$

(81)

Moreover, based on Lemma 1:

$$\begin{aligned} \varvec{\bar{x}}_{i}^{new}=\underset{\scriptstyle \varvec{\bar{x}}_{i}^* \in \overline{\mathbb {X}}_i}{{\text {argmin}}} \displaystyle \int _{\overline{\mathbb {X}}_i} \dfrac{n_i}{n_i+1}\text {Tr}\left( \varvec{R}_{t_i}\left( \varvec{Y}_i^\text {obs} \right) \right) v_i\left( \varvec{\bar{x}}_{i}\vert \varvec{\overline{X}}_{i}^{\text {obs}},\varvec{\bar{x}}_{i}^* \right) d\mu _{\overline{\mathbb {X}}_i}\left( \varvec{\bar{x}}_{i} \right) . \end{aligned}$$

(82)

By noting that $\text {Tr}\left( \varvec{R}_{t_i}\left( \varvec{Y}_i^\text {obs} \right) \right)$ does not depends on $\varvec{\bar{x}}_{i}$ and $\varvec{\bar{x}}_{i}^*$, the criterion can finally be written:

$$\begin{aligned} \varvec{\bar{x}}_{i}^{new}=\underset{\scriptstyle \varvec{\bar{x}}_{i}^* \in \overline{\mathbb {X}}_i}{{\text {argmin}}} \displaystyle \int _{\overline{\mathbb {X}}_i} v_i\left( \varvec{\bar{x}}_{i}\vert \varvec{\overline{X}}_{i}^{\text {obs}},\varvec{\bar{x}}_{i}^* \right) d\mu _{\overline{\mathbb {X}}_i}\left( \varvec{\bar{x}}_{i} \right) . \end{aligned}$$

(83)

1.4 Proof of Proposition 3

By definition (see Eq. 47) and given the independence of $\varvec{Y}_2$ and $\varvec{Y}_3$, one has:

$$\begin{aligned} \varvec{Y}^c_\text {nest}\left( \varvec{x}_1,\varvec{x}_2 \right) \overset{d}{=}\varvec{\mu }_{2}^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) +\varvec{\xi }_{2}\sigma _{\varvec{x}2}^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) , \end{aligned}$$

where $\varvec{\xi }_2$ and $\varvec{\xi }_3$ are independent Gaussian variables, such that $\varvec{\xi }_2 \sim \mathcal {N}\left( \varvec{0},\widehat{\varvec{R}}_{t_2} \right)$ and $\varvec{\xi }_3 \sim \mathcal {N}\left( \varvec{0},\widehat{\varvec{R}}_{t_3} \right)$.

This implies:

$$\begin{aligned} \mathbb {E}\left[ \varvec{Y}^c_\text {nest}\left( \varvec{x}_1,\varvec{x}_2 \right) \right] =\mathbb {E}\left[ \varvec{\mu }_2^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) \right] , \end{aligned}$$

because $\varvec{\xi }_2$ and $\varvec{\xi }_3$ are independent and $\mathbb {E}\left[ \varvec{\xi }_2 \right] =\varvec{0}$.

Besides, one has:

$$\begin{aligned}&\varvec{Y}^c_\text {nest}\left( \varvec{x}_1,\varvec{x}_2 \right) \varvec{Y}^c_\text {nest}\left( \varvec{x}_1,\varvec{x}_2 \right) ^T\\&\quad =\varvec{\mu }_2^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) \varvec{\mu }_2^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) ^T\\&\qquad +\,\left( \varvec{\sigma }_2^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) \right) ^2 \varvec{\xi }_2\varvec{\xi }_2^T\\&\qquad +\,\varvec{\sigma }_2^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) \varvec{\xi }_2\varvec{\mu }_2^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) ^T\\&\qquad +\,\varvec{\sigma }_2^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) \varvec{\mu }_2^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) \varvec{\xi }_2^T. \end{aligned}$$

Given that $\varvec{\xi }_2$ and $\varvec{\xi }_3$ are independent, $\mathbb {E}\left[ \varvec{\xi }_2 \right] =\varvec{0}$ and $\mathbb {E}\left[ \varvec{\xi }_2\varvec{\xi }_2^T \right] =\widehat{\varvec{R}}_{t_2}$, one has:

$$\begin{aligned} \begin{array}{c} \mathbb {E}\left[ \varvec{Y}^c_\text {nest}\left( \varvec{x}_1,\varvec{x}_2 \right) \varvec{Y}^c_\text {nest}\left( \varvec{x}_1,\varvec{x}_2 \right) ^T \right] = \mathbb {E}\left[ \left( \varvec{\sigma }_2^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) \right) ^2 \right] \widehat{\varvec{R}}_{t_2} \\ +\,\mathbb {E}\left[ \varvec{\mu }_2^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) \varvec{\mu }_2^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) ^T \right] . \end{array} \end{aligned}$$

1.5 Proof of Proposition 4

By definition (see Eq. 47), one has:

$$\begin{aligned} \varvec{Y}^c_\text {nest}\left( \varvec{x}_1,\varvec{x}_2 \right) \overset{d}{=}\varvec{\mu }_{2}^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) +\varvec{\xi }_{2}\sigma _{\varvec{x}2}^c\left( \varvec{\mu }_{3}^c\left( \varvec{x}_1 \right) +\varvec{\xi }_{3}\sigma _{\varvec{x}3}^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) , \end{aligned}$$

where $\varvec{\xi }_2 \sim \mathcal {N}\left( \varvec{0},\widehat{\varvec{R}}_{t_2} \right)$ and $\varvec{\xi }_3 \sim \mathcal {N}\left( \varvec{0},\widehat{\varvec{R}}_{t_3} \right)$.

If $\sigma ^c_{\varvec{x}3}\left( \varvec{x}_1 \right)$ is small enough, then the previous equation can be linearized with respect to $\sigma ^c_{\varvec{x}3}\left( \varvec{x}_1 \right)$. Thus, one has:

$$\begin{aligned}&\varvec{Y}^c_\text {nest}\left( \varvec{x}_1,\varvec{x}_2 \right) \overset{d}{\approx }\varvec{\mu }_2^c\left( \varvec{\mu }_3^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) +\dfrac{\partial \varvec{\mu }_2^c}{\partial \varvec{\rho } }\left( \varvec{\mu }^c_3\left( \varvec{x}_1 \right) ,\varvec{x}_2 \right) \sigma ^c_{\varvec{x}3}\left( \varvec{x}_1 \right) \varvec{\xi }_3 \\&\quad +\,\varvec{\xi }_2\sigma ^c_{\varvec{x}2}\left( \varvec{\mu }_3^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) . \end{aligned}$$

Thanks to the fact that $\varvec{\xi }_2$ and $\varvec{\xi }_3$ are independent, a Gaussian predictor of the nested code can be obtained from the previous equation. Furthermore, this Gaussian process has the following mean function:

$$\begin{aligned} \varvec{\mu }_\text {nest}^c\left( \varvec{x}_1,\varvec{x}_2 \right) =\varvec{\mu }_2^c\left( \varvec{\mu }_3^c\left( \varvec{x}_1 \right) , \varvec{x}_2 \right) , \end{aligned}$$

(84)

and its covariance function is:

$$\begin{aligned}&\varvec{C}^c_\text {nest}\left( \left( \varvec{x}_1,\varvec{x}_2 \right) ,\left( \varvec{x}_1^{'},\varvec{x}_2^{'} \right) \right) = \widehat{\varvec{R}}_{t_2}C^c_{2}\left( \left( \varvec{\mu }^c_3\left( \varvec{x}_1 \right) ,\varvec{x}_2 \right) , \left( \varvec{\mu }^c_3\left( \varvec{x}_1^{'} \right) ,\varvec{x}_2^{'} \right) \right) \\&\quad +\,\dfrac{\partial \varvec{\mu }_2^c}{\partial \varvec{\rho } }\left( \varvec{\mu }^c_3\left( \varvec{x}_1 \right) ,\varvec{x}_2 \right) \widehat{\varvec{R}}_{t_3} \left( \dfrac{\partial \varvec{\mu }_2^c}{\partial \varvec{\rho } }\left( \varvec{\mu }^c_3\left( \varvec{x}_1^{'} \right) ,\varvec{x}_2^{'} \right) \right) ^T C^c_{3}\left( \varvec{x}_1,\varvec{x}_1^{'} \right) . \end{aligned}$$