Multiplier bootstrap methods for conditional distributions

Abstract

The multiplier bootstrap is a fast and easy-to-implement alternative to the standard bootstrap; it has been used successfully in many statistical contexts. In this paper, resampling methods based on multipliers are proposed in a general framework where one investigates the stochastic behavior of a random vector $\mathbf {Y}\in \mathbb {R}^d$ conditional on a covariate $X \in \mathbb {R}$. Specifically, two versions of the multiplier bootstrap adapted to empirical conditional distributions are introduced as alternatives to the conditional bootstrap and their asymptotic validity is formally established. As the method walks hand-in-hand with the functional delta method, theory around the estimation of statistical functionals is developed accordingly; this includes the interval estimation of conditional mean and variance, conditional correlation coefficient, Kendall’s dependence measure and copula. Composite inference about univariate and joint conditional distributions is also considered. The sample behavior of the new bootstrap schemes and related estimation methods are investigated via simulations and an illustration on real data is provided.

Appendices

Appendix 1: Assumptions on the weights

The following notation will be adopted in the sequel. For a sequence of random variables $(Z_n)_{n\in \mathbb {N}}$ and a sequence of real numbers $(a_n)_{n\in \mathbb {N}}$, one writes $Z_n = O_e(a_n)$ if there exist finite constants $C_1, C_2$ and $\eta > 0$ such that $\mathrm{P}( Z_n/a_n \ge C_1 ) \le C_2 \exp ( - n^\eta / C_1 )$. Also, let $J_{nx}= [ \, \min _{i \in I_{nx}} \, x_i ,\max _{i \in I_{nx}} \, x_i \, ]$, where $I_{nx} = \{ j \in \{ 1, \ldots , n \} : w_{nj}(x,h_n) > 0 \}$. The assumptions on the weights listed below are for fixed designs; the adaptation to random designs generally consists in replacing o by $o_{{\text {a.s.}}}$ and O by $O_{\text {a.s.}}$, except for $W_4$, $W_6$, $W'_2$ and $W''_2$, where O has to be replaced by $O_e$. This stronger requirement ensures the validity of Lemma 3 and Lemma 4 in Omelka et al. (2013) in the context of a random design. These lemmas will prove useful in the proofs presented in the online resource.

Assumptions $W_1$–$W_5$ below are needed to establish the asymptotic behavior of $\mathbb {F}_{xh}$ stated in Sect. 2.1; they are also needed, as well as $W_6$, in the proof of Proposition 1.

$W_1$.:

$\displaystyle \sqrt{n h_n} \max _{1 \le i \le n} \left| w_{ni}(x,h_n) \right| = o(1)$;

$W_2$.:

$\displaystyle \sqrt{n h_n} \left| \sum _{i=1}^n w_{ni}(x,h_n)(x_i-x) - h_n^2 \, K_2 \right| = o(1)$ for some $K_2 = K_2(x) \in (0,\infty )$;

$W_3$.:

$\displaystyle \sqrt{n h_n} \left| \sum _{i=1}^n w_{ni}(x,h_n) (x_i-x)^2 / 2 - h_n^2 \, K_3 \right| = o(1)$ for some $K_3 = K_3(x) \in (0,\infty )$;

$W_4$.:

$\displaystyle n h_n \sum _{i=1}^n \{ w_{ni}(x,h_n) \}^2 - K_4 = O(1)$ for some $K_4 = K_4(x) \in (0,\infty )$;

$W_5$.:

$\displaystyle \max _{i \in I_{nx}} x_i - \min _{i \in I_{nx}} x_i = o(1)$;

$W_6$.:

$\displaystyle (n h_n)^2 \sum _{i=1}^n \{ w_{ni}(x,h_n) \}^4 = O(n^{-\delta })$ for some $\delta > 0$.

The next assumptions are needed in order to establish the consistency of ${\widehat{B_x}}$ and the validity of the two multiplier bootstrap methods.

$W'_1$.:

$\displaystyle \left| \sum _{i=1}^n w'_{ni}(x,g_n) \right| = o(1)$;

$W'_2$.:

$\displaystyle n^\delta \sum _{i=1}^n \{ w'_{ni}(x,g_n) \}^2 = O(1)$ for some $\delta >0$;

$W'_3$.:

$\displaystyle \left| \sum _{i=1}^n w'_{ni}(x,g_n)(x_i - x) - 1 \right| = o(1)$;

$W''_1$.:

$\displaystyle \sup _{z \in J_{nx}} \left| \sum _{i=1}^n w''_{ni}(z,g_n) \right| = o(1)$;

$W''_2$.:

$\displaystyle n^\delta \sup _{z \in J_{nx}} \sum _{i=1}^n \{ w''_{ni}(x,g_n) \}^2 = O(1)$ for some $\delta >0$;

$W''_3$.:

$\displaystyle \sup _{z \in J_{nx}} \left| \sum _{i=1}^n w''_{ni}(z,g_n)(x_i-x) \right| = o(1)$;

$W''_4$.:

$\displaystyle \sup _{z \in J_{nx}} \left| \sum _{i=1}^n \{ w''_{ni}(x,g_n)(x_i-x) \}^2 - 1 \right| = o(1)$;

$W''_5$.:

$\displaystyle g_n^2 \sup _{z \in J_{nx}} \sum _{i=1}^n \left| w''_{ni}(z,g_n) \right| = O(1)$;

$W''_6$.:

One can find constants $C_1, C_2 < \infty$ and $\alpha >0$ such that for all $z_1,z_2 \in J_{nx}$,

$$\begin{aligned} \max _{i \in I_{nx}} \left| w''_{ni}(z_1,g_n) - w''_{ni}(z_2,g_n) \right| \le C_1 \, g_n^{-C_2} |z_1-z_2|^\alpha . \end{aligned}$$

Appendix 2: Complementary computations

1.1 Hadamard derivative of the variance functional

Let $\mathcal {D}$ be the space of distribution functions. Then, for any $\delta \in \mathcal {D}$, define $\delta _t = \delta + t {\varDelta }_t \in \ell ^\infty (\mathbb {R})$ such that ${\varDelta }_t \rightarrow {\varDelta }\in \mathbb {D}_0$ uniformly as $t\rightarrow 0$. Then,

$\{ \delta _t(y_1\wedge y_2) - \delta _t(y_1)\delta _t(y_2) \} - \{ \delta (y_1\wedge y_2) - \delta (y_1)\delta (y_2) \}$

$$\begin{aligned}= & {} t {\varDelta }_t(y_1\wedge y_2) - t \left\{ {\varDelta }_t(y_1)\delta (y_2) - {\varDelta }_t(y_2)\delta (y_1) \right\} \\&- \, t^2 {\varDelta }_t(y_1){\varDelta }_t(y_2) \\= & {} t \left\{ \mathbb {I}(y_1\le y_2) - \delta (y_2) \right\} {\varDelta }_t(y_1) \\&+ \, t \left\{ \mathbb {I}(y_2\le y_1) - \delta (y_1) \right\} {\varDelta }_t(y_2) - \, t^2 {\varDelta }_t(y_1){\varDelta }_t(y_2). \end{aligned}$$

It then follows that as $t\rightarrow 0$,

$$\begin{aligned} \frac{{\varLambda }^v(\delta _t) - {\varLambda }^v(\delta )}{t}= & {} 2 \int _{\mathbb {R}^2} \left\{ \mathbb {I}(y_1\le y_2)- \delta (y_2) \right\} {\varDelta }_t(y_1) \mathrm{d}y_1 \mathrm{d}y_2 \\&- \, t \left\{ \int _{\mathbb {R}} {\varDelta }_t(y_1) \mathrm{d}y_1 \right\} ^2 \\= & {} 2 \int _{\mathbb {R}^2} \left\{ \mathbb {I}(y_1\le y_2)- \delta (y_2) \right\} {\varDelta }_t(y_1) \mathrm{d}y_1 \mathrm{d}y_2 \\&- \, t \left\{ {\varLambda }^m({\varDelta }_t) \right\} ^2 \\\rightarrow & {} 2 \int _{\mathbb {R}^2} \left\{ \mathbb {I}(y_1\le y_2)- \delta (y_2) \right\} {\varDelta }(y_1) \mathrm{d}y_1 \mathrm{d}y_2. \end{aligned}$$

1.2 Conditional variance and correlation

First introduce the notation $\mathbf{A}^{(j)} = ( Y_{j1} - \theta _{xjh}^\mathrm{m}, \ldots , Y_{jn} - \theta _{xjh}^\mathrm{m} )$ and $\mathbf{B}^{(jj')} = \mathrm{diag} ( (\mathbf{A}^{(j)})^\top \mathbf{A}^{(j')})$ for $j,j' \in \{1,2\}$. For the variance functional, one has

$\displaystyle {\widehat{({\varLambda }^\mathrm{v})'_{F_x}}}({\varDelta }) = ({\varLambda }^\mathrm{v})_{F_{xh}}'({\varDelta }) =$

$$\begin{aligned} 2 \sum _{i=1}^n w_{ni}(x,h_n) \int _{\mathbb {R}^2} \left\{ \mathbb {I}(y_2 \le y_1) - \mathbb {I}(Y_i \le y_1) \right\} {\varDelta }(y_2) \, \mathrm{d}y_1 \mathrm{d}y_2, \end{aligned}$$

and then long but straightforward computations yield ${\widehat{{\varTheta }}}_x^{(1)} = {\widehat{{\varTheta }}}_x^{(2)} = \sqrt{n h_n} \, \mathbf{B}^{(11)} \, \mathbf{L}_x$. For the correlation functional, one can show that ${\varLambda }^\mathrm{Cov}(F_{xh}) = \mathbf{B}^{(12)} \, \mathbf{w}_x^\top$, ${\varLambda }^\mathrm{v}(F_{1xh}) = \mathbf{B}^{(11)} \, \mathbf{w}_x^\top$ and ${\varLambda }^\mathrm{v}(F_{2xh}) = \mathbf{B}^{(22)} \, \mathbf{w}_x^\top$, so that one can write $\rho _{xh} = \mathbf{B}^{(12)} \, \mathbf{w}_x^\top / \{ \mathbf{B}^{(11)} \, \mathbf{w}_x^\top \}^{1/2} \{ \mathbf{B}^{(22)} \, \mathbf{w}_x^\top \}^{1/2}$. By similar computations, one can show that

$$\begin{aligned} {\widehat{{\varTheta }}}_x^{(1)}= & {} ({\varLambda }^\rho )'_{F_{xh}}({\widehat{\mathbb {F}}}_x) \\= & {} \rho _{xh} \left( { \mathbf{B}^{(12)} \, \mathbf{L}_x^\top \over \mathbf{B}^{(12)} \, \mathbf{w}_x^\top } - { \mathbf{B}^{(11)} \, \mathbf{L}_x^\top \over 2 \mathbf{B}^{(11)} \, \mathbf{w}_x^\top } - { \mathbf{B}^{(22)} \, \mathbf{L}_x^\top \over 2 \mathbf{B}^{(22)} \, \mathbf{w}_x^\top } \right) \end{aligned}$$

and

$$\begin{aligned} {\widehat{{\varTheta }}}_x^{(2)}= & {} \sqrt{n h_n} \left\{ { \mathbf{B}^{(12)} \left( \mathbf{w}_x + \mathbf{L}_x \right) ^\top \over \{ \mathbf{B}^{(11)} \left( \mathbf{w}_x + \mathbf{L}_x \right) ^\top \mathbf{B}^{(22)} \left( \mathbf{w}_x + \mathbf{L}_x \right) ^\top \}^{1/2} } \right. \\&\left. -\, { \mathbf{B}^{(12)} \, \mathbf{w}_x^\top \over \{ \mathbf{B}^{(11)} \, \mathbf{w}_x^\top \mathbf{B}^{(22)} \, \mathbf{w}_x^\top \}^{1/2} } \right\} . \end{aligned}$$

1.3 Kendall’s tau

For $\mathbf {Y}_i = (Y_{1i},Y_{2i})$, $i \in \{ 1, \ldots , n \}$, let $(\mathcal {K}_{ii'}) = \mathbb {I}(\mathbf {Y}_i \le \mathbf {Y}_{i'})$. Then, one obtains from a direct computation that $\tau _{xh} = {\varLambda }^\tau (F_{xh}) = -1 + 4 \, \mathbf{w}_x \, \mathcal {K} \, \mathbf{w}_x^\top$. Next, since $\mathrm{d}F_{xh}$ puts mass $w_{ni}(x,h_n)$ at $\mathbf {Y}_i$, one obtains

$\displaystyle ({\varLambda }^\tau )_{F_{xh}}'({\varDelta })$

$$\begin{aligned}= & {} 4 \int _{\mathbb {R}^2} {\varDelta }(y_1,y_2) \, \mathrm{d}F_{xh}(y_1,y_2) + 4 \int _{\mathbb {R}^2} F_{xh}(y_1,y_2) \, \mathrm{d}{\varDelta }(y_1,y_2) \\= & {} 4 \sum _{i=1}^n w_{ni}(x,h_n) \left\{ {\varDelta }(Y_{1i},Y_{2i}) + \int _{Y_{1i}}^\infty \int _{Y_{2i}}^\infty \mathrm{d}{\varDelta }(y_1,y_2) \right\} . \end{aligned}$$

If $\lim _{\gamma \rightarrow \infty } {\varDelta }(\gamma ,y) = \lim _{\gamma \rightarrow \infty } {\varDelta }(y,\gamma ) = 0$, the last equality reduces to

$$\begin{aligned} ({\varLambda }^\tau )_{F_{xh}}'({\varDelta }) = 8 \sum _{i=1}^n w_{ni}(x,h_n) \, {\varDelta }(Y_{1i},Y_{2i}). \end{aligned}$$

Hence,

$$\begin{aligned} {\widehat{{\varTheta }}}_x^{(1)} = ({\varLambda }^\tau )'_{F_{xh}}({\widehat{\mathbb {F}}}_x) = 8 \sqrt{n h_n} \, \mathbf{w}_x \, \mathcal {K} \, \mathbf{L}_x^\top . \end{aligned}$$

Also, one can show that ${\varLambda }^\tau ({\widehat{F}}_x) = - 1 + 4 ( \mathbf{w}_x + \mathbf{L}_x ) \, \mathcal {K} \, ( \mathbf{w}_x + \mathbf{L}_x )^\top$, and then

$$\begin{aligned} {\widehat{{\varTheta }}}_x^{(2)}= & {} \sqrt{n h_n} \left\{ \left( - 1 + 4 ( \mathbf{w}_x + \mathbf{L}_x ) \, \mathcal {K} \, ( \mathbf{w}_x + \mathbf{L}_x )^\top \right) \right. \\&\left. - \, \left( 4 \, \mathbf{w}_x \, \mathcal {K} \, \mathbf{w}_x^\top - 1 \right) \right\} \\= & {} 4 \sqrt{n h_n} \left( \mathbf{w}_x \, \mathcal {K} \, \mathbf{L}_x^\top + \mathbf{L}_x \, \mathcal {K} \, \mathbf{w}_x^\top + \mathbf{L}_x \, \mathcal {K} \, \mathbf{L}_x^\top \right) . \end{aligned}$$

1.4 Copula

The empirical conditional copula is $C_{xh}(\mathbf {u}) = {\varLambda }^\mathrm{C}(F_{xh}) = \mathcal {I}^\mathrm{C}(\mathbf {u}) \, \mathbf{w}_x^\top$, where $(\mathcal {I}^\mathrm{C})_i = \mathbb {I}\{ \mathbf {Y}_i \le \mathbf{F}_{xh}^{-1}(\cdot )\}$. One also obtains ${\widehat{{\varTheta }}}_x^{(1)} = {\widehat{{\varTheta }}}_x^{(1)}(\mathbf {u}) = {\widetilde{\mathcal {I}}}^\mathrm{C}(\mathbf {u}) \, \mathbf{L}_x^\top$, where

$$\begin{aligned} ( {\widetilde{\mathcal {I}}}^\mathrm{C}(\mathbf {u}) )_i= & {} \mathbb {I}\left\{ \mathbf {Y}_i \le \mathbf{F}_{xh}^{-1}(\mathbf {u}) \right\} \\&- \, \sum _{j=1}^d {\widehat{C_x^{(j)}}}(\mathbf {u}) \, \mathbb {I}\left\{ Y_{ji} \le F_{xjh}^{-1}(u_j) \right\} . \end{aligned}$$

The formula for ${\widehat{{\varTheta }}}_x^{(2)}$ is more involved.

1.5 Testing a covariate’s influence

Let $(\mathcal {Y}_{ii'})_{i,i'=1}^n = Y_i \wedge Y_{i'}$. Then by straightforward computations, one can show that

$$\begin{aligned} S_{xx'h}^\mathrm{CvM} = n h_n \left( \mathbf{w}_x - \mathbf{w}_{x'} \right) \, \mathcal {Y} \, \left( \mathbf{w}_x - \mathbf{w}_{x'} \right) ^\top \end{aligned}$$

and

$$\begin{aligned} {\widehat{S}}_{xx'}^\mathrm{CvM} = n h_n \left( \mathbf{L}_x - \mathbf{L}_{x'} \right) \, \mathcal {Y} \, \left( \mathbf{L}_x - \mathbf{L}_{x'} \right) ^\top . \end{aligned}$$