Conditional independence testing via weighted partial copulas

Abstract

The test statistic proposed in this paper is an explicit Cramér–von Mises transformation of a certain weighted partial copula function. The regions of rejection are computed using a bootstrap procedure which mimics conditional independence by generating samples from the product measure of the estimated conditional marginals. Under certain (high-level) conditions (on the estimated conditional marginals), rates of convergence for the weighted partial copula process and the test statistic as well as the weak convergence under the null of the normalized test statistic are established. These high-level conditions on the estimated margins are shown to be valid in a variety of examples ranging from nonparametric kernel to linear quantile regression estimates. Finally, an experimental section demonstrates that the proposed test has competitive power compared to recent state-of-the-art methods such as kernel-based test.

Introduction

Let $(Y_1,Y_2,\mathit{X})\in \mathbb{R}\times \mathbb{R}\times {\mathbb{R}}^d$ be a triple of continuous random variables. We say that $Y_1$ and $Y_2$ are conditionally independent given $\mathit{X}$ if $\forall (y_1,y_2,\mathit{x})\in \mathbb{R}\times \mathbb{R}\times {\mathbb{R}}^d$:

$$Pr(Y_1\le y_1,Y_2\le y_2\mid \mathit{X}=\mathit{x})=Pr(Y_1\le y_1\mid \mathit{X}=\mathit{x})Pr(Y_2\le y_2\mid \mathit{X}=\mathit{x}).$$

This property is denoted by $Y_1\perp \perp Y_2\mid \mathit{X}$ and roughly speaking, it means that for a given value of $\mathit{X}$, the knowledge of $Y_1$ does not provide any further information on $Y_2$ (and vice versa). Determining conditional independence has become in the recent years a fundamental question in statistics and machine learning. For instance, it plays a key role in defining graphical models [1], [39]; see also [45] for a study specific to cellular networks. Moreover the concept of conditional independence lies at the core of sufficient dimension reduction methods [42] and is useful to conduct variable selection in regression [41]. Finally, conditional independence is relevant in many application fields such as economy [32] or psychometry [2].

The approach taken in this paper is related to the well-studied problem of (unconditional) independence testing, in which, inspired by [35], rank-based statistics have received an increasing interest [13], [52], [53], [54]. Because they do not depend on the marginals, rank-based statistics have became a key tool for modeling the joint distribution of random variables without being affected by their margins. A most natural quantity that measures the dependency between random variables is the empirical copula function, defined as the estimated joint cumulative distribution between the ranks. It has been used with success in independence testing [22], [23] and we refer to [20], [55] for additional details on the estimation of the copula function. A common difficulty when using the empirical copula is that the distribution (even the asymptotic one) of the resulting statistic is unknown and making difficult the computation of quantiles used to build the rejection regions. To overcome this issue, bootstrap procedures have been of prime interest because they allow, using simulated data, to approximate the statistic distribution [8], [20], [49].

The conditional copula of $Y_1$ and $Y_2$ given $\mathit{X}$ is defined in the same way as the copula of $Y_1$ and $Y_2$ but uses the conditional distribution of $Y_1$ and $Y_2$ given $\mathit{X}$ in place of the joint distribution of $Y_1$ and $Y_2$. Compared to the copula, the conditional copula captures the conditional dependency between random variables and is thus useful to build conditional dependency measures [15], [26]. Therefore, as in the case of independence testing, the conditional copula appears to be a relevant tool for building a statistical test of conditional independence. This has been pointed out as an “interesting open issue” in [62, Section4].

In this work, a new statistical test procedure, called the weighted partial copula test is investigated to assess conditional independence. The proposed approach follows from using the weighted partial copula, an integral transform allowing a simple characterization of conditional independence. Given estimators of the conditional marginals of $Y_1$ and $Y_2$ given $\mathit{X}$, the empirical weighted partial copula is introduced to estimate the weighted partial copula and the test statistic results from an easy-to-compute Cramér–von Mises transformation. The use of the weighted partial copula is motivated by the conditional moment restrictions literature (see [40] and the reference therein) in which integrated criteria, similar to the weighted partial copula, have been frequently used. Those criteria are interesting because even when they involve local estimates converging at a slower rate than $1/\sqrt{n}$, their convergence rates are in many cases in $1/\sqrt{n}$.

Inspired by the independence testing literature [3], [38], the computation of the quantiles is made using a bootstrap procedure which generates bootstrap samples from the product of the marginal estimators to mimic the null hypothesis. Thanks to this bootstrap procedure, one is allowed to perform the weighted partial copula test using any marginal estimates as soon as one can generate random variables from these margins.

The theoretical results of the paper are as follows. Under the uniform convergence of the estimated marginals, convergence rates are established for the empirical weighted partial copula and for the resulting test statistic. These results are interesting because they allow, in principle, to use any reasonable estimates of the margins when running the test. Moreover, under an additional assumption on the estimated quantiles, which is satisfied for standard parametric estimate such as in linear quantile regression [36], the weak convergence of the test statistic is obtained.

Nonparametric testing for conditional independence between continuous variables has received an increasing interest the past few years [43]. Some of the existing approaches are based on comparing the (estimated) conditional distributions involved in the definition of conditional independence. The probability distributions can be compared using their conditional characteristic functions as in Su and White, Wang et al. [59], [63], their conditional densities as proposed in [60], or their conditional copulas as studied in [7]. Unfortunately, the estimation of these conditional quantities are subjected to the well-known curse of dimensionality, i.e., the convergence rates are badly affected by the dimension of the conditioning variable. As a consequence, the power of the previous tests rapidly deteriorates if the conditioning variable has a large dimension. Note also Bergsma [4] that uses partial copulas to derive the test statistic. Unfortunately, partial copulas fail to capture the whole conditional distribution and lead to detect a null hypothesis much larger than conditional independence. Other approaches rely on the characterization of conditional independence using cross-covariance operators defined on reproducing kernel Hilbert spaces [21]. Extending the Hilbert–Schmidt independence criterion proposed in Gretton et al., Zhang et al. [27], [65] defines a kernel-based conditional independence test (KCI-test) by estimating the cross-covariance operator (see also [58]). A surge of recent research [16], [51], [56] has focused on testing conditional independence using permutation-based tests. The work of [9] had led to many conditional independence tests depending on the availability of an approximation to the distribution of $Y_j|\mathit{X}$, $j=1,2$, such as the conditional permutation test proposed in [5].

While it is impossible to claim the superiority of our approach compared to the existing methods, we may emphasize several notable advantages:

• (flexibility) Any marginal estimate can be used to compute the test statistic. Since the test accuracy depends on the approximation quality of the marginal distribution, the test shall benefit from a precise estimation procedure of the margins.

• (bootstrap quantiles) A nice feature of the proposed framework is that the quantiles of the test can be approximated using the bootstrap as soon as random generation from each margins is available.

• (convergence rate and dimension) In certain situations (parametric and smoothed nonparametric marginal estimate), the rate of convergence of the test statistic under the null is $1/n$ and hence is free from the covariates dimension (whereas it impacts most nonparametric procedures). This property suggests that the proposed test has reasonable power in the multidimensional context.

The outline is as follows. In Section 2, we introduce the weighted partial copula test and provide implementation details regarding the bootstrap procedure. In Section 3, we state the main theoretical results. In Section 4, the proposed approach is compared to several competitors on simulated datasets. The mathematical proofs are gathered in the Appendix.