On some consistent tests of mutual independence among several random vectors of arbitrary dimensions

Abstract

Testing for mutual independence among several random vectors is a challenging problem, and in recent years, it has gained significant attention in statistics and machine learning literature. Most of the existing tests of independence deal with only two random vectors, and they do not have straightforward generalizations for testing mutual independence among more than two random vectors of arbitrary dimensions. On the other hand, there are various tests for mutual independence among several random variables, but these univariate tests do not have natural multivariate extensions. In this article, we propose two general recipes, one based on inter-point distances and the other based on linear projections, for multivariate extensions of these univariate tests. Under appropriate regularity conditions, these resulting tests turn out to be consistent whenever we have consistency for the corresponding univariate tests. We carry out extensive numerical studies to compare the empirical performance of these proposed methods with the state-of-the-art methods.

Appendix

Lemma 1

For any fixed $\delta >0$, define $p_{n,\sigma }(\delta ,F)= \sup _{\mathbf{a}} \Pr (\left| {{\mathbb {T}}}_{n,\sigma }(F^{\mathbf{a}})-{\mathbb T}_{\sigma }(F^{\mathbf{a}})\right| >\delta )$. If m(n) is a polynomial function of n, then $\sum _{n=1}^\infty m(n)p_{n,\sigma }(\delta ,F)<\infty$.

Proof

Recall that ${{\mathbb {T}}}_{\sigma }(F^{\mathbf{a}})$ and ${\mathbb T}_{n,\sigma }(F^{\mathbf{a}})$ are defined as

$$\begin{aligned} {\mathbb T}_{\sigma }(F^{\mathbf{a}})=\frac{\gamma _{\sigma }(C^{\mathbf{a}},\varPi )}{\gamma _{\sigma }(M,\varPi )} ~\text{ and }~~ {\mathbb T}_{n,\sigma }=\frac{\gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma }(M_n,\varPi _n)}, \end{aligned}$$

where $C^{\mathbf{a}}$, $M$, $\varPi$ and $C^{\mathbf{a}}_n$, $M_n$, $\varPi _n$ are as defined in Section 2.2.

Now, observe that

$$\begin{aligned}&\Pr (\left| {{\mathbb {T}}}_{n,\sigma }(F^\mathbf{a})-{\mathbb T}_\sigma (F^\mathbf{a})\right|>\delta ) \\&\quad =\Pr \left( \left| \frac{\gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma }(M_n,\varPi _n)}-\frac{\gamma _{\sigma }(C^{\mathbf{a}},\varPi )}{\gamma _{\sigma }(M,\varPi )}\right|>\delta \right) \\&\quad \le \text {Pr}\left( \left| \frac{\gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma }(M_n,\varPi _n)}-\frac{\gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma }(M,\varPi )}\right|>\delta /2\right) \\&\qquad +\text {Pr}\left( \left| \frac{\gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma }(M,\varPi )}-\frac{\gamma _{\sigma }(C^{\mathbf{a}},\varPi )}{\gamma _{\sigma }(M,\varPi )}\right| >\delta /2\right) . \end{aligned}$$

First consider the term

$$\begin{aligned}&\left| \frac{\gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma }(M_n,\varPi _n)}-\frac{\gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma }(M,\varPi )}\right| \\&\quad =\frac{\gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma }(M_n,\varPi _n)\gamma _{\sigma }(M,\varPi )}~\times \left| \gamma _{\sigma }(M_n,\varPi _n)-\gamma _{\sigma }(M,\varPi )\right| . \end{aligned}$$

Note that ${\gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)}/\big ({\gamma _{\sigma }(M_n,\varPi _n)\gamma _{\sigma }(M,\varPi )}\big )$ is uniformly bounded and $\left| \gamma _{\sigma }(M_n,\varPi _n)-\gamma _{\sigma }(M,\varPi )\right|$ is a non-random quantity that converges to 0 as n tends to infinity (see Roy et al. 2020, Lemma L2). Therefore, there exists $n_0\ge 1$ such that for all $n >n_0$,

$$\begin{aligned}&\left| \frac{\gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma }(M_n,\varPi _n)}-\frac{\gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma }(M,\varPi )}\right| < \delta /2, ~~\text{ i.e., }\\&\text {Pr}\left( \left| \frac{\gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma }(M_n,\varPi _n)}-\frac{\gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma }(M,\varPi )}\right| >\delta /2\right) =0. \end{aligned}$$

Note that this $n_0$ does not depend on $\mathbf{a}$. Again,

$$\begin{aligned}&\text {Pr}\left( \left| \frac{\gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma }(M,\varPi )}-\frac{\gamma _{\sigma }(C^{\mathbf{a}},\varPi )}{\gamma _{\sigma }(M,\varPi )}\right|>\delta /2\right) \\&\quad =\text {Pr}\left( \left| \gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)-\gamma _{\sigma }(C^{\mathbf{a}},\varPi )\right| >\delta ^{*}\right) , \end{aligned}$$

where $\delta ^*=\gamma _{\sigma }(M,\varPi )\frac{\delta }{2}$ is a positive constant. So, it is enough to show the finiteness of

$$\begin{aligned} \sum _{n=1}^{\infty } m(n) \sup _{\mathbf{a}}\text {Pr}\Bigl (\left| \gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)-\gamma _{\sigma }(C^{\mathbf{a}},\varPi )\right| >\delta ^{*}\Bigr ). \end{aligned}$$

Let $F^{(\mathbf{a},1)},\ldots ,F^{(\mathbf{a},p)}$ be the distribution functions of $X^{(\mathbf{a},1)},\ldots ,X^{(\mathbf{a},p)}$, respectively. Also define $C^{\mathbf{a}*}_n$ to be the empirical joint distribution of $\left( F^{(\mathbf{a},1)}(X^{(\mathbf{a},1)}_1), \ldots ,\right.$$\left. F^{(\mathbf{a},p)}(X^{(\mathbf{a},p)}_1)\right) ,\ldots ,\left( F^{(\mathbf{a},1)}(X^{(\mathbf{a},1)}_n),\ldots ,F^{(\mathbf{a},p)}(X^{(\mathbf{a},p)}_n)\right) .$ Then, following Theorem 6 of Roy et al. (2020), we get

$$\begin{aligned}&\text {Pr}\Bigl (\left| \gamma _{\sigma }(C^{\mathbf{a}}_n,\varPi _n)-\gamma _{\sigma }(C^{\mathbf{a}},\varPi )\right|>\delta ^{*}\Bigr )\\&\quad \le \text {Pr}\left( \left| \gamma _{\sigma }^2(C^{\mathbf{a}}_n,\varPi _n)-\gamma _{\sigma }^2(C^{\mathbf{a}}_n,\varPi )\right| ^{\frac{1}{2}}\right. \\&\qquad +\gamma _{\sigma }(C^{\mathbf{a}}_n,C^{\mathbf{a}*}_n)+\gamma _{\sigma }(C^{\mathbf{a}*}_n,C^{\mathbf{a}})>\delta ^{*}\Bigr )\\&\quad \le \text {Pr}\left( \left| \gamma _{\sigma }^2(C^{\mathbf{a}}_n,\varPi _n)-\gamma _{\sigma }^2(C^{\mathbf{a}}_n,\varPi )\right| ^{\frac{1}{2}}>\delta ^{*}/3\right) \\&\qquad +\text {Pr}\Bigl (\gamma _{\sigma }(C^{\mathbf{a}}_n,C^{\mathbf{a}*}_n)>\delta ^{*}/3\Bigr )+\text {Pr}\Bigl (\gamma _{\sigma }(C^{\mathbf{a}*}_n,C^{\mathbf{a}})>\delta ^{*}/3\Bigr ). \end{aligned}$$

Now, from Lemma L2 in Roy et al. (2020), we can show that there exists $n_1\ge 1$ ($n_1$ does not depend on $\mathbf{a}$) such that for all $n>n_1$, $\text {Pr}\left( \left| \gamma _{\sigma }^2(C^{\mathbf{a}}_n,\varPi _n)-\gamma _{\sigma }^2(C^{\mathbf{a}}_n,\varPi )\right| ^{\frac{1}{2}}> \delta ^{*}/3\right) =0$. Again, from Lemmas L3 and L4 in Roy et al. (2020), we have $\text {Pr}\Bigl (\gamma _{\sigma }(C^{\mathbf{a}}_n,C^{\mathbf{a}*}_n)>\delta ^{*}/3\Bigr )<2p\exp \left( -\frac{n{\delta ^*}^2}{18pL^2}\right)$ and $\text {Pr}\left( \gamma _{\sigma }(C^{\mathbf{a}*}_n,C^{\mathbf{a}})>\delta ^{*}/{3}\right) <\exp \left( -\frac{n}{2}\left( \frac{\delta ^{*}}{3}-\frac{2}{\sqrt{n}}\right) ^2\right)$, respectively, where L is a positive constant independent of $\mathbf{a}$. So, to prove the lemma, it is sufficient to show that

$$\begin{aligned}&\sum _{n=1}^{\infty } m(n) \left[ 2p\exp \left( -\frac{n{\delta ^*}^2}{18pL^2}\right) \right. \\&\qquad \left. +\exp \left( -\frac{n}{2}\left( \frac{\delta ^{*}}{3}-\frac{2}{\sqrt{n}}\right) ^2\right) \right] <\infty . \end{aligned}$$

Clearly, this is true since m(n) is a polynomial function of n. $\square$

Lemma 2

Consider a sequence $\{\sigma (n): n\ge 1\}$, which converges to $\sigma _0>0$. For any fixed $\delta >0$, define ${\tilde{p}}_n(\delta ,F)=\sup _{\mathbf{a}} \Pr (\left| {\mathbb T}_{n,\sigma (n)}(F^{\mathbf{a}})-{\mathbb T}_{\sigma _0}(F^{\mathbf{a}})\right| >\delta )$. If m(n) is a polynomial function of n, then $\sum _{n=1}^\infty m(n){\tilde{p}}_{n}(\delta ,F)<\infty$.

Proof

Note that for any fixed $\mathbf{a}$,

$$\begin{aligned}&\Pr (\left| {{\mathbb {T}}}_{n,\sigma (n)}(F^{\mathbf{a}})-{\mathbb T}_{\sigma _0}(F^{\mathbf{a}})\right|>\delta )\\&\quad \le \Pr (\left| {{\mathbb {T}}}_{n,\sigma (n)}(F^{\mathbf{a}})-{\mathbb T}_{n,\sigma _0}(F^{\mathbf{a}})\right|>\delta /2)\\&\qquad +\Pr (\left| {{\mathbb {T}}}_{n,\sigma _0}(F^{\mathbf{a}})-{\mathbb T}_{\sigma _0}(F^{\mathbf{a}})\right| >\delta /2). \end{aligned}$$

So, in view of Lemma 1, it is enough to show that $\sum _{n=1}^{\infty } m(n) \sup _{\mathbf{a}} \Pr (\left| {\mathbb T}_{n,\sigma (n)}(F^{\mathbf{a}})-{\mathbb T}_{n,\sigma _0}(F^{\mathbf{a}})\right| >\delta /2)$ is finite. Now, note that

$$\begin{aligned}&\left| {{\mathbb {T}}}_{n,\sigma (n)}(F^{\mathbf{a}})-{{\mathbb {T}}}_{n,\sigma _0}(F^{\mathbf{a}})\right| \\&\quad =\left| \frac{\gamma _{\sigma (n)}(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma (n)}(M_n,\varPi _n)}-\frac{\gamma _{\sigma _0}(C^{\mathbf{a}}_n,\varPi _n)}{\gamma _{\sigma _0}(M_n,\varPi _n)}\right| \\&\quad \le \gamma _{\sigma (n)}(C^{\mathbf{a}}_n,\varPi _n) \times \left| \frac{1}{\gamma _{\sigma (n)}(M_n,\varPi _n)}-\frac{1}{\gamma _{\sigma _0}(M_n,\varPi _n)}\right| \\&\quad \quad +\frac{|\gamma _{\sigma (n)}(C^{\mathbf{a}}_n,\varPi _n)-\gamma _{\sigma _0}(C^{\mathbf{a}}_n,\varPi _n)|}{\gamma _{\sigma _0}(M_n,\varPi _n)}\\&\quad =A_n + B_n, ~\text{(say) }. \end{aligned}$$

Note that while the term $\gamma _{\sigma (n)}(C^{\mathbf{a}}_n,\varPi _n)$ is uniformly bounded, $\left| \frac{1}{\gamma _{\sigma (n)}(M_n,\varPi _n)}-\frac{1}{\gamma _{\sigma _0}(M_n,\varPi _n)}\right|$ is a non-random quantity converging to 0 (follows from Lemma L6 in Roy et al. (2020)). Therefore, there exists a natural number $n_0$ (independent of $\mathbf{a}$) such that for all $n> n_0$, we have $A_n \le \delta /4$ with probability one.

Now, $\gamma _{\sigma _0}(M_n,\varPi _n)$ is a non-random quantity converging to $\gamma _{\sigma _0}(M,\varPi )$. Also, from Lemma L6 in Roy et al. (2020), we get a non-random upper bound for $|\gamma _{\sigma (n)}(C^{\mathbf{a}}_n,\varPi _n)-\gamma _{\sigma _0}(C^{\mathbf{a}}_n,\varPi _n)|$, which converges to 0. Since this upper bound does not depend on $\mathbf{a}$, we get a natural number $n_1$ (independent of $\mathbf{a}$) such that for all $n> n_1$, $B_n \le \delta /4$ with probability one.

Using these two facts, for all $n>\max \{n_0,n_1\}$, we have $A_n+B_n < \delta /2$ with probability one, and hence,

$$\begin{aligned} \Pr \Big (\left| {{\mathbb {T}}}_{n,\sigma (n)}(F^{\mathbf{a}})-{\mathbb T}_{n,\sigma _0}(F^{\mathbf{a}})\right| >\delta /2\Big )=0. \end{aligned}$$

This implies the finiteness of $\sum _{n=1}^{\infty } m(n) \sup _{\mathbf{a}} \Pr (|{\mathbb T}_{n,\sigma (n)}(F^{\mathbf{a}})-{\mathbb T}_{n,\sigma _0}(F^{\mathbf{a}})|>\delta /2)$. $\square$