Weighted multiple testing procedure for grouped hypotheses with k-FWER control

Abstract

In this paper, k-FWER (generalized familywise error rate) control for grouped hypotheses testing is considered. We offer the weights for the p-values in each group, by maximizing an objective function, which is the expectation of the proportion of rejected hypotheses. This objective function utilizes not only the information of the proportion of true null hypotheses, but also the null and non-null distributions of the p-values in each group. When this information is known prior, our weighted testing procedure controls k-FWER for arbitrarily dependent p-values. When this information is unknown, and is estimated from the data, our procedure asymptotically controls k-FWER under the weak dependence assumption of the p-values in each group. The new procedure is shown to be more powerful than some existing procedures both in theory and simulations. For illustration, the proposed procedure is applied to analyse the adequate yearly progress data.

Appendices

Appendix

Proof of Proposition 1

Proof

$$\begin{aligned} power(\varvec{\omega }^*,\frac{k}{n}\alpha )&=\frac{1}{n_{T1}}\sum _{g=1}^G\sum _{j\in I_{g1}}P\left( p_{gj}\le \frac{k\alpha }{na_{T0}}\right) \\&=\sum _{g=1}^G\frac{n_{g1}}{n_{T1}}F\left( \frac{k\alpha }{na_{T0}}\right) \\&=F\left( \frac{k\alpha }{na_{T0}}\right) , \end{aligned}$$

and

$$\begin{aligned} power(\varvec{\omega }^{**},\frac{k}{n}\alpha )&=\frac{1}{n_{T1}}\sum _{g=1}^G\sum _{j\in I_{g1}}P\left( p_{gj}\le \frac{k\alpha }{na_{g0}}\right) \nonumber \\&=\sum _{g=1}^G\frac{n_{g1}}{n_{T1}}F\left( \frac{k\alpha }{na_{g0}}\right) \\&\ge F\left( \frac{k\alpha }{n\sum _{g=1}^G\frac{n_{g1}}{n_{T1}}a_{g0}}\right) ,\nonumber \end{aligned}$$

(5)

where the last inequality follows from the convexity of $F(\cdot /x)$, and the equality holds if and only if $a_{10}=\cdots =a_{G0}$. Thus, it only needs to verify $na_{T0}\ge n\sum _{g=1}^G\frac{n_{g1}}{n_{T1}}a_{g0}$. Since

$$\begin{aligned}&na_{T0}- n\sum _{g=1}^G\frac{n_{g1}}{n_{T1}}a_{g0}\\&\quad =\sum _{g=1}^Gn_{g0}-\frac{n}{n_{T1}}\sum _{g=1}^{G}n_{g1}a_{g0}\\&\quad =\sum _{g=1}^Gn_{g0}-\frac{n}{n-\sum _{g=1}^Gn_{g0}}\sum _{g=1}^{G}(n_g-n_{g0})\frac{n_{g0}}{n_g}\\&\quad =\frac{\sum _{g=1}^Gn_{g}\sum _{g=1}^{G}\frac{n_{g0}^2}{n_g}-(\sum _{g=1}^Gn_{g0})^2}{n-\sum _{g=1}^Gn_{g0}}\ge 0, \end{aligned}$$

where the last inequality follows from Cauchy-Schwartz inequality, and the equality holds if and only if $n_{10}/n_1=\cdots =n_{G0}/n_G$, e.g. $a_{10}=\cdots =a_{G0}$.

Thus, the proof is completed. $\square$

Remark 1

In the proof of Theorem 2 in Zhao (2014), $power(\varvec{\omega }^{**},\frac{k}{n}\alpha )=\sum _{g=1}^Ga_{g1}F(\frac{k\alpha }{na_{g0}})$, which is not true, and we correct in formula (5).

1.1 Proof of Theorem 1

Proof

Denote V be the number of false rejections.

$$\begin{aligned} k\text {-}\mathrm {FWER}&\le \frac{E(V)}{k}\\&=\frac{1}{k}E\left\{ \sum _{g=1}^G\sum _{j\in I_{g0}}I\left( p_{gj}\le \omega _{g} \frac{k}{n}\alpha \right) \right\} \\&=\frac{n}{k}\sum _{g=1}^Ga_ga_{g0}F_{g0}\left( \omega _g\frac{k}{n}\alpha \right) \\&=\frac{n}{k}\sum _{g=1}^Ga_ga_{g0}\omega _g\frac{k}{n}\alpha \\&=\frac{n}{k}\frac{k}{n}\alpha =\alpha . \end{aligned}$$

$\square$

1.2 Proof of Theorem 2

Proof

For simplicity, we drop the subscript n in $V_n$ and $k_n$ throughout the proof.

(1). Since $\frac{k}{n}\alpha \rightarrow c\alpha$, following the proof of Lemma 3 in Zhao and Zhang (2014), we can get

$$\begin{aligned} {\hat{r}}\left( \varvec{\omega }_0\left( {\hat{r}}, \frac{k}{n}\alpha \right) ,\frac{k}{n}\alpha \right) {\mathop {\rightarrow }\limits ^{\mathrm {P}}}{\tilde{r}}(\varvec{\omega }_0({\tilde{r}}, c\alpha ),c\alpha ), \text { as } n\rightarrow \infty , \end{aligned}$$

and

$$\begin{aligned} {\widehat{power}}\left( \varvec{\omega }_0\left( {\hat{r}}, \frac{k}{n}\alpha \right) ,\frac{k}{n}\alpha \right) {\mathop {\rightarrow }\limits ^{\mathrm {P}}}\widetilde{power}(\varvec{\omega }_0({\tilde{r}}, c\alpha ),c\alpha ), \text { as } n\rightarrow \infty . \end{aligned}$$

(2) Denote $\varvec{\omega }_0({\hat{r}}, \frac{k}{n}\alpha )=(\omega _{10}({\hat{r}}, \frac{k}{n}\alpha ),\ldots ,\omega _{G0}({\hat{r}}, \frac{k}{n}\alpha ))'$, and $\varvec{\omega }_0({\tilde{r}}, c\alpha )=(\omega _{10}({\tilde{r}}, c\alpha ),\ldots ,\omega _{G0}({\tilde{r}}, c\alpha ))'$. Similarly,

$$\begin{aligned} \frac{1}{n}\sum _{g=1}^G\sum _{j\in I_{g0}}I\left( p_{gj}\le \omega _{g0}({\hat{r}}, \frac{k}{n}\alpha )\frac{k}{n}\alpha \right) {\mathop {\rightarrow }\limits ^{\mathrm {P}}}\sum _{g=1}^G\pi _g\pi _{g0}F_{g0}\left( \omega _{g0}({\tilde{r}}, c\alpha )c\alpha \right) . \end{aligned}$$

By dominated convergence theorem, we have

$$\begin{aligned} E\left[ \frac{1}{n}\sum _{g=1}^G\sum _{j\in I_{g0}}I\left( p_{gj}\le \omega _{g0}({\hat{r}}, \frac{k}{n}\alpha )\frac{k}{n}\alpha \right) \right] \rightarrow \sum _{g=1}^G\pi _g\pi _{g0}F_{g0}\left( \omega _{g0}({\tilde{r}}, c\alpha )c\alpha \right) . \end{aligned}$$

Then for WT$(\varvec{\omega }_0({\hat{r}}, \frac{k}{n}\alpha ),\frac{k}{n}\alpha )$,

$$\begin{aligned} k\text {-}\mathrm {FWER}&\le \frac{E(V)}{k}\\&=\frac{n}{k}E\left\{ \frac{1}{n}\sum _{g=1}^G\sum _{j\in I_{g0}}I\left( p_{gj}\le \omega _{g0}({\hat{r}}, \frac{k}{n}\alpha )\frac{k}{n}\alpha \right) \right\} \\&=\left[ \frac{1}{c}+o(1)\right] \left[ \sum _{g=1}^G\pi _g\pi _{g0}F_{g0}\left( \omega _{g0}({\tilde{r}}, c\alpha )c\alpha \right) +o(1)\right] \\&=\frac{1}{c}\sum _{g=1}^G\pi _g\pi _{g0}\omega _{g0}({\tilde{r}}, c\alpha )c\alpha +o(1)\\&=\alpha +o(1). \end{aligned}$$

Immediately, $\limsup _{n\rightarrow \infty } P(V\ge k)\le \alpha .$

(3) Since ${\hat{\pi }}_{g0}{\mathop {\rightarrow }\limits ^{\mathrm {P}}}\pi _{g0}\in (0,1)$, ${\hat{\pi }}_{T0}=\sum _{g=1}^Ga_g{\hat{\pi }}_{g0}{\mathop {\rightarrow }\limits ^{\mathrm {P}}}\pi _{T0}=\sum _{g=1}^G\pi _g\pi _{g0}$, and $\frac{1}{{\hat{\pi }}_{T0}}\frac{k}{n}\alpha {\mathop {\rightarrow }\limits ^{\mathrm {P}}}\frac{1}{\pi _{T0}}c\alpha$. It is easy to see

$$\begin{aligned} \frac{1}{n}\sum _{g=1}^G\sum _{j\in I_{g0}}I\left( p_{gj}\le \frac{1}{{\hat{\pi }}_{T0}}\frac{k}{n}\alpha \right) {\mathop {\rightarrow }\limits ^{\mathrm {P}}}\sum _{g=1}^G\pi _g\pi _{g0}F_{g0}\left( \frac{1}{\pi _{T0}}c\alpha \right) . \end{aligned}$$

By dominated convergence theorem, we have

$$\begin{aligned} E\left[ \frac{1}{n}\sum _{g=1}^G\sum _{j\in I_{g0}}I\left( p_{gj}\le \frac{1}{{\hat{\pi }}_{T0}}\frac{k}{n}\alpha \right) \right] \rightarrow \sum _{g=1}^G\pi _g\pi _{g0}F_{g0}\left( \frac{1}{\pi _{T0}}c\alpha \right) . \end{aligned}$$

As similar as that for WT$(\varvec{\omega }_0({\hat{r}}, \frac{k}{n}\alpha ),\frac{k}{n}\alpha )$, we can get

$$\begin{aligned} k\text {-}\mathrm {FWER}\le \alpha +o(1), \end{aligned}$$

which means $\limsup _{n\rightarrow \infty } P(V\ge k)\le \alpha .$

(4) Since $\frac{1}{{\hat{\pi }}_{g0}}\frac{k}{n}\alpha {\mathop {\rightarrow }\limits ^{\mathrm {P}}}\frac{1}{\pi _{g0}}c\alpha$ for all g, the proof is similar, and is omitted.

$\square$