Inferring Boolean functions via higher-order correlations

Abstract

Both the Walsh transform and a modified Pearson correlation coefficient can be used to infer the structure of a Boolean network from time series data. Unlike the correlation coefficient, the Walsh transform is also able to represent higher-order correlations. These correlations of several combined input variables with one output variable give additional information about the dependency between variables, but are also more sensitive to noise. Furthermore computational complexity increases exponentially with the order. We first show that the Walsh transform of order 1 and the modified Pearson correlation coefficient are equivalent for the reconstruction of Boolean functions. Secondly, we also investigate under which conditions (noise, number of samples, function classes) higher-order correlations can contribute to an improvement of the reconstruction process. We present the merits, as well as the limitations, of higher-order correlations for the inference of Boolean networks.

Appendix

In this section the intricate relationship between functional analysis on Boolean functions and correlation will be further elaborated. It has been shown that Pearson correlation and Fourier expansion with basis functions of order 1 are sufficient to reconstruct the dependencies of a Boolean network if that network only consists of monotone functions. We now give a formulation for the influence of a variable in terms of partial discrete derivatives to highlight the connection of monotonicity to spectral coefficients. In this context, the modified Pearson correlation can also be interpreted as the marginal effect of a variable $x_i$ on $f$ as measured via a simple linear regression relating $x_i$ to $f$.

Influence of a variable: We define the $i$th partial discrete derivative of a Boolean function $f$ at $\varvec{x}$ with respect to $x_i$ similar to, but more general than Gotsman and Linial (1994) as

$$\begin{aligned} \partial _i(f,\varvec{x},x_i) = \frac{ f(\varvec{x}|_{i:x_i}) - f(\varvec{x}|_{i:\overline{x}_i}) }{x_i - \overline{x}_i}, \quad \mathrm{with} ~ \overline{x}_i = \fancyscript{B} {\setminus } x_i. \end{aligned}$$

Hence we can depict the influence of a variable $X_i$ as mean absolute derivative of a Boolean function $f$ with respect to $X_i$:

$$\begin{aligned} \mathrm{I}_{i,\fancyscript{D}}(f) = E_{\fancyscript{D}} [ | \partial _i(f,\varvec{X},X_i) |]. \end{aligned}$$

Utilizing the Fourier expansion (4) the influence can be written as

$$\begin{aligned} \mathrm{I}_{i,\fancyscript{D}}(f) = E_{\fancyscript{D}} \left[ \left|\sum \limits _{ S :i \in S} \hat{f}(S) \chi _{S \setminus i}(\varvec{X}) \right| \right]. \end{aligned}$$

If there is a mean linear tendency of a function $f$ projected to a dimension $i$, it is possible to access the relevant variable $x_i \in \mathrm{rel}(f)$ via linear regression, as the expected value is not zero. It may not be possible to further simplify this mean analytically for all classes of Boolean functions, but for monotone functions we can show the following connection:

Linear regression on monotonicity: Any Boolean function decomposed in (4) as

$$\begin{aligned} f(\varvec{x})= g(\varvec{x}) + x_i \cdot h(\varvec{x}),\quad \mathrm{with} ~ \varvec{x} \in \{-1,+1\}^n \end{aligned}$$

is a multivariate polynomial over the reals. A function $f$ is monotone in a variable $x_i$, if either

$$\begin{aligned} f(\varvec{x}_{|i:-1}) \le f(\varvec{x}_{|i:+1}) \quad \mathrm{or} \quad f(\varvec{x}_{|i:-1}) \ge f(\varvec{x}_{|i:+1}) \end{aligned}$$

holds for all $\varvec{x}$. A Boolean function monotone increasing in variable $i$ holds

$$\begin{aligned} f(\varvec{x}_{|i:-1}) = g(\varvec{x}) - h(\varvec{x}) \le f(\varvec{x}_{|i:+1}) = g(\varvec{x}) + h(\varvec{x}), \end{aligned}$$

which implies that the partial discrete derivative is $\partial _i(f,\varvec{x},1) \ge 0$. For functions, monotone decreasing in dimension $i$, the derivative is $\partial _i(f,\varvec{x},1) \le 0$. As the sign of $\partial _i$ is not alternating, the computation of the absolute value and the mean can be interchanged and the influence can be expressed as

$$\begin{aligned} \mathrm{I}_{i,\fancyscript{D}}(f) = | r_{\fancyscript{D}}(i,f) | = \left| \sum \limits _{ S :i \in S} \hat{f}(S) \prod \limits _{j \in S \setminus i} \mu _j \right| ,\quad \mathrm{with}~ \mu _j = E[x_j], \end{aligned}$$

(8)

namely the absolute regression coefficient of the linear Boolean model in variable $i$. As suggested in Sect. 2, monotonicity of a variable is sufficient to access the relevance of a variable via linear regression interpreted as influence. This fact holds for general product distributions.

We recall some facts for Boolean functions on uniform distributed domain (subscript $\fancyscript{U}$), also mentioned in (Gotsman and Linial 1994): The influence of a variable $i$ is lower-bounded as follows:

$$\begin{aligned} \mathrm{I}_{i,\fancyscript{U}}(f) \ge \max \limits _{ S: i \in S } ~ | \hat{f} (S) |. \end{aligned}$$

For Boolean functions which are monotonic in a variable $i$ the definitions in (Kahn et al. 1988) further imply $\text{ I}_{i,\fancyscript{U}}(f) = | \hat{f} (\{i\}) |,$ meaning that the linear coefficient of this variable $i$ is the largest absolute value.

Limits on Pearson correlation: If the monotonicity on a variable $i$ is not given by premise the interpretation of the absolute correlation coefficient $r_{\fancyscript{D}}(i,f)$ as influence $\text{ I}_{i,\fancyscript{D}}(f)$ is not given (see 8). Nevertheless the relevance of the variable can be addressed via modified Pearson correlation if the correlation coefficient differs from zero. A non-zero coefficient simply indicates that there is a (linear) dependency (see Covariance in 5) between this variable $i$ and the value of the Boolean function. The absolute value of the correlation is limited by non-linearities of the Boolean function and by the probability distribution of the variables, more precisely by those variables which are not directly considered in the correlation.its “structure“, the Fourier coefficients $\hat{f}$ at uniform distribution and the means of the Fourier basis under general product distribution $\fancyscript{D}$.