On the Decision Tree Complexity of Threshold Functions

Abstract

In this paper we study decision tree models with various types of queries. For a given function it is usually not hard to determine the complexity in the standard decision tree model (each query evaluates a variable). However in more general settings showing tight lower bounds is substantially harder. Threshold functions often have non-trivial complexity in such models and can be used to provide interesting examples. Standard decision trees can be viewed as a computational model in which each query depends on only one input bit. In the first part of the paper we consider natural generalization of standard decision tree model: we address decision trees that are allowed to query any function depending on two input bits. We show the first lower bound of the form n − o(n) for an explicit function (namely, the majority function) in this model. We also show that in the decision tree model with AND and OR queries of arbitrary fan-in the complexity of the majority function is n − 1. In the second part of the paper we address parity decision trees that are allowed to query arbitrary parities of input bits. There are various lower bound techniques for parity decision trees complexity including analytical techniques (degree over $\mathbb {F}_{2}$, Fourier sparsity, granularity) and combinatorial techniques (generalizations of block sensitivity and certificate complexity). These techniques give tight lower bounds for many natural functions. We give a new inductive argument tailored specifically for threshold functions. A combination of this argument with granularity lower bound allows us to provide a simple example of a function for which all previously known lower bounds are not tight.

Appendix:

1.1 A.1 Proof of Theorem 1

We start with an upper bound. The following lemma is a simple adaptation of the folklore algorithm (see, e.g. [22]).

Lemma 11

$$ \mathsf{D}_{\oplus}(\text{{MAJ}}_{n}) \leq n - \mathsf{B}(n)+1. $$

Proof

Our parity decision tree will mostly make queries of the form y ⊕ z for a pair of variables. Note that such a query basically checks whether y and z are equal.

Our algorithm will maintain splitting of input variables into blocks of two types. We will maintain the following properties:

• the size of each block is a power of 2;

• all variables in each block of type 1 are equal;

• blocks of type 2 are balanced, that is they have equal number of ones and zeros.

In the beginning of the computation each variable forms a separate block of size one. During each step the algorithm will merge two blocks into a new one. Thus, after k steps the number of blocks is n − k.

The algorithm works as follows. On each step we pick two blocks of type 1 of equal size. We pick one variable from each block and query the parity of these two variables. If the variables are equal, we merge the blocks into a new block of type 1. If the variables are not equal, the new block is of type 2. The process stops when there are no blocks of type 1 of equal size.

It is easy to see that all of the properties listed above are maintained. In the end of the process we have some blocks of the second type (possibly none of them) and some blocks of the first type (possibly none of them) of pairwise non-equal size. Note that the value of the majority function is determined by the value of variables in the largest block of type 1. Indeed, all blocks of type 2 are balanced and the largest block of type 1 has more variables then all other blocks of type 1 in total. Thus, to find the value of MAJ_n it remains to query one variable from the largest block of type 1. Note, that the case when there are no blocks of type 1 in the end of the process correspond to balanced input (and even n). In this case we can tell that the output is − 1 without any additional queries.

Note that the sum of sizes of all blocks is equal to n. Since the size of each block is a power of 2, there are at least B(n) blocks in the end of the computation (one cannot break n in the sum of less then B(n) powers of 2). Thus, overall we make at most n −B(n) + 1 queries and the lemma follows. □

Before proceeding with the lower bound through granularity we briefly discuss lower bounds that can be obtained by other approaches. It is known that spar(MAJ_n) = 2^n− 1 [19]. Thus from the sparsity lower bound we can only get $\mathsf {D}_{\oplus }(\text {{MAJ}}_{n}) \geq \log \mathsf {spar}(\text {{MAJ}}_{n})/2 = \frac {n-1}{2}$.

Note also that each input x ∈{0, 1}ⁿ to MAJ_n lies in the constant-valued subcube of dimension at least $\lceil \frac {n-1}{2}\rceil$. Indeed, if MAJ_n(x) = 1 just pick a subcube on some subset of variables of size $\lceil \frac {n-1}{2}\rceil$ containing all ones of the input. The case MAJ_n(x) = − 1 is symmetric. Thus, in the approach through certificate complexity we get $\mathsf {D}_{\oplus }(\text {{MAJ}}_{n}) \geq \lceil \frac {n-1}{2}\rceil$.

Finally, we observe that the degree approach also does not give a matching lower bound.

Lemma 12

For any n we have $\deg (\text {{MAJ}}_{n})=2^{p}$ where p is the largest integer such that 2^p ≤ n.

Proof

Consider a multilinear polynomial p over $\mathbb {F}_{2}$ computing MAJ_n. For a set $S \subseteq [n]$ denote by c_S the coefficient of the monomial ${\prod }_{i\in S} x_{i}$ in p. Denote |S| = k and denote by $x_{S} \in \{0,1\}^{n}$ the input such that x_i = 1 iff i ∈ S. By [12, Section 2.1] we have

$$ c_{S} = \bigoplus_{x \leq x_{S}} \text{{MAJ}}_{n}(x), $$

where the order on {0, 1}ⁿ is coordinate-wise.

From this we obtain that

$$ c_{S} = {\sum}_{i=\lceil \frac{n}{2}\rceil}^{k} \binom{k}{i} = {\sum}_{i=\lceil \frac{n}{2}\rceil}^{k} (-1)^{i}\binom{k}{i} \pmod 2, $$

where the second equation follows since changing the sign of an integer summand does not change its remainder when divided by 2.

Denote $l = \lceil \frac {n}{2}\rceil$. We can simplify the latter sum as follows:

$$ {\sum}_{i=l}^{k} (-1)^{i}\binom{k}{i} = {\sum}_{i=l}^{k} (-1)^{i} \left( \binom{k-1}{i-1} + \binom{k-1}{i} \right) = (-1)^{l} \binom{k-1}{l-1}. $$

By Kummer’s theorem $\binom {k-1}{l-1}$ is odd iff the summation process of l − 1 and k − l in binary representation does not have any carry bits. Note that both $l-1=\lceil \frac n2 \rceil -1$ and $k-l\leq \lfloor \frac n2 \rfloor$ are less or equal n/2. Thus their binary representations are one bit shorter than the binary representation of n. The maximal k for which $\binom {k-1}{l-1}$ is odd (and thus c_S is non-zero) is the one for which k − l has a binary representation inverted compared to l − 1, that is (k − l) + (l − 1) = k − 1 has a binary representation consisting of ones only. That is, k is a power of 2 not exceeding n. □

It is not hard to see that this lower bound matches the upper bound of Lemma 11 only for n = 2^r and n = 2^r + 1. On the other hand, for example it is far from optimal by approximately a factor of 2 for n = 2^r − 1 for some r.

We next show that Lemma 3 gives a tight lower bound for parity decision tree complexity of MAJ_n.

Lemma 13

gran(MAJ_n) = n −B(n).

Proof

We will show that gran(MAJ_n) ≥ n −B(n). The inequality in the other direction follows from Lemma 11 and Lemma 3.

We consider the Fourier coefficient $\widehat {\text {{MAJ}}}_{n}([n])$ and show that its granularity is at least n −B(n). Let k = ⌊(n + 1)/2⌋. Note that k is the smallest number such that MAJ_n is − 1 on inputs with k ones.

Then we have

$$ \begin{array}{@{}rcl@{}} \widehat{\text{{MAJ}}}_{n}([n]) &= \frac{1}{2^{n}}\left( {\sum}_{i=0}^{k-1} (-1)^{i}\binom{n}{i} - {\sum}_{i=k}^{n} (-1)^{i}\binom{n}{i}\right)\\ &= \frac{1}{2^{n}}\left( {\sum}_{i=0}^{n} (-1)^{i}\binom{n}{i} - 2 {\sum}_{i=k}^{n} (-1)^{i}\binom{n}{i}\right) = \frac{1}{2^{n}} \left( 0 - 2 {\sum}_{i=k}^{n} (-1)^{i}\binom{n}{i}\right). \end{array} $$

From this we can see that

$$ \mathsf{gran}(\widehat{\text{{MAJ}}}_{n}([n])) =n - \mathsf{P}\left( 2{\sum}_{i=k}^{n} (-1)^{i}\binom{n}{i}\right). $$

We proceed to simplify the sum of binomials (a very similar analysis is presented in [22]):

$$ {\sum}_{i=k}^{n} (-1)^{i}\binom{n}{i} = {\sum}_{i=k}^{n} (-1)^{i} \left( \binom{n-1}{i-1} + \binom{n-1}{i} \right) = (-1)^{k} \binom{n-1}{k-1}. $$

Thus it remains to compute $\mathsf {P}(2\binom {n-1}{k-1})$. For even n = 2h we have k = h and $2\binom {n-1}{k-1} = 2\binom {2h-1}{h-1} = \binom {2h}{h}$. For odd n = 2h + 1 we have k = h + 1 and $2\binom {n-1}{k-1} = 2\binom {2h}{h}$.

By [22, Proposition 3.4] we have $\mathsf {P}(\binom {2h}{h})=\mathsf {B}(h)$ (alternatively this can be seen from Kummer’s theorem). Finally, notice that B(2h) = B(h) and B(2h + 1) = B(h) + 1. It follows that

$$ \mathsf{P}\left( 2{\sum}_{i=k}^{n} (-1)^{i}\binom{n}{i}\right) = \mathsf{B}(n) $$

and

$$ \mathsf{gran}(\widehat{\text{{MAJ}}}_{n}([n])) = n - \mathsf{B}(n). $$

□

Overall, Theorem 1 follows.

1.2 A.2 Proof of Theorem 2

We start with an upper bound.

Lemma 14

$\mathsf {D}_{\oplus }(\text {{MAJ}}^{\otimes k}_3) \leq (n+1)/2$.

Proof

Basically, recursive majority $\text {{MAJ}}^{\otimes k}_3$ is a function computed by a Boolean circuit whose graph is a complete ternary tree of depth k, each internal vertex is labeled by the function MAJ₃ and each leaf is labeled by a (fresh) variable.

To construct an algorithm we first generalize the problem. We consider functions computed by Boolean circuits whose graphs are ternary tree, where each non-leaf has fan-in 3 and is labeled by MAJ₃, and each leaf is labeled by a fresh variable. We will show that if the number of non-leaf vertices in the circuit is l, then the function can be computed by a parity decision tree of size l + 1.

The proof is by induction on l. If l = 1, then the function in question is just MAJ₃ and by the results of Appendix A it can be computed by a parity decision tree of depth 2.

For the step of induction consider a tree with l non-leaf vertices. Consider a non-leaf vertex of the largest depth. All of its three inputs must be variables, lets denote them by y, z and t, and in this vertex the function MAJ₃(y,z,t) is computed. Our first query will be y ⊕ z. It will tell us whether y and z are equal. If y = z are equal, then MAJ₃(y,z,t) = y, and if y≠z, then MAJ₃(y,z,t) = t. Thus, we can substitute the gate in our vertex by the corresponding variable and reduce the problem to the circuit with l − 1 non-leaf vertices. By induction hypothesis, the function computed by this circuit can be computed by at most (l − 1) + 1 = l queries. Thus, our original function is computable by l + 1 queries.

It is left to observe that a complete ternary tree of depth k has $3^{k-1}+\ldots +1=\frac {3^{k}-1}{2}$ non-leaf vertices and for this tree our algorithm makes $\frac {3^{k}+1}{2}=\frac {n+1}{2}$ queries. □

Before proceeding to the lower bound through granularity we again discuss lower bounds that can be obtained by other techniques.

First note that each input x ∈{0, 1}ⁿ lies in the subspace of co-dimension at most 2^k on which the function is constant. For this it is enough to show that in each x we can flip 3^k − 2^k variables without changing the value of the function. This is easy to check by induction on k. For k = 1 there are two variables that are equal to each other and we can flip the third variable without changing the value of the function. For k > 1 consider inputs to the MAJ₃ at the top of the circuit. Two of them are equal and by induction hypothesis we can flip 3^k− 1 − 2^k− 1 variables in each of them without changing the value of the function. The last input to the top gate does not affect the value of the function and we can flip all 3^k− 1 variables in it. Overall this gives us 3^k − 2^k variables. This gives us $\mathsf {C}_{\oplus }(\text {{MAJ}}^{\otimes k}_3) \leq 2^{k} = n^{\log _{3} 2}$ which does not give a matching lower bound.

Also note that the polynomial computing MAJ₃ is p(x) = x₁x₂ ⊕ x₂x₃ ⊕ x₁x₃. The polynomial for $\text {{MAJ}}^{\otimes k}_3$ can be computed by a simple composition of p with itself. It is easy to see that its degree is $2^{k} = n^{\log _{3} 2}$. Thus, an approach through polynomials over $\mathbb {F}_{2}$ does not give strong lower bounds.

For Fourier analytic considerations it is convenient to switch to {− 1, 1} Boolean inputs. For a variable y ∈{0, 1} let us denote by $y^{\prime }\in \{-1,1\}$ the variable $y^{\prime }=1-2y$. For now we will use new variables as inputs to Boolean functions.

The Fourier decomposition of MAJ₃ is

$$ \begin{array}{@{}rcl@{}} \text{{MAJ}}_{3}(y^{\prime},z^{\prime},t^{\prime}) = \frac{1}{2}\left( y^{\prime}+z^{\prime}+t^{\prime} - y^{\prime}z't^{\prime}\right). \end{array} $$

(2)

From this the Fourier decomposition of $\text {{MAJ}}^{\otimes k}_3$ can be obtained by recursion:

$$ \begin{array}{@{}rcl@{}} \begin{array}{lllll} \text{{MAJ}}^{\otimes k}_3(x^{1},x^{2},x^{3}) = & \frac{1}{2} (\text{{MAJ}}^{\otimes k-1}_3(x^{1})+\text{{MAJ}}^{\otimes k-1}_3(x^{2})+\text{{MAJ}}^{\otimes k-1}_3(x^{3})\\ & - \text{{MAJ}}^{\otimes k-1}_3(x^{1})\cdot\text{{MAJ}}^{\otimes k-1}_3(x^{2})\cdot\text{{MAJ}}^{\otimes k-1}_3(x^{3})), \end{array} \end{array} $$

(3)

where x¹,x²,x³ are blocks of 3^k− 1 variables.

Lemma 2 can give lower bounds up to n/2 and thus in principle might give at least almost matching lower bound. However, this is not the case as we discuss below.

Note that since there is no free coefficient in the polynomial (2), Fourier coefficients arising from all three summands in the right-hand side of (3) will not cancel out with each other: no two of them have equal set of variables. Thus, if we denote $S(k)=\mathsf {spar}(\text {{MAJ}}^{\otimes k}_3)$ we have that S(1) = 4 and

$$ S(k) = 3S(k-1) + S(k-1)^{3} $$

(4)

for k > 1. On one hand, this means that S(k) > S(k − 1)³. This gives $S(k) > 2^{2\cdot 3^{k-1}}$. Thus $\log \mathsf {spar}(\text {{MAJ}}^{\otimes k}_3) > 2\cdot 3^{k-1}= 2n/3$ and $\mathsf {D}_{\oplus }(\text {{MAJ}}^{\otimes k}_3)> n/3$.

On the other hand if we let $S^{\prime }(k)=S(k)+1/2$, it is easy to check that (4) implies

$$ S^{\prime}(k) < S^{\prime}(k-1)^{3}. $$

Since $S^{\prime }(1)=9/2$ this gives $S^{\prime }(k) < 2^{(\log _{2}\frac {9}{2}) \cdot 3^{k-1}}$. Thus,

$$ \log \mathsf{spar}(\text{{MAJ}}^{\otimes k}_3) < \left( \log_{2}\frac{9}{2}\right) \cdot \frac{n}{3} < 0.723 \cdot n. $$

Thus Lemma 2 can give us a lower bound of at most 0.362 ⋅ n. We note that this upper bound on the sparsity can be further improved by letting $S^{\prime }(k) = S(k)+\alpha$ for smaller α.

Now we proceed to the tight lower bound. Again we will estimate gran $(\widehat {\text {{MAJ}}}^{\otimes k}_{3}[n])$. Observe that this Fourier coefficient can be easily computed from (2) and (3). Indeed, from (2) we have that $\left |\widehat {\text {{MAJ}}}^{\otimes 1}_{3}[n]\right | = \frac 12$. From (3) we have that

$$ \left|\widehat{\text{{MAJ}}}^{\otimes k}_{3}[n]\right| = \left|\frac{1}{2}(\widehat{\text{{MAJ}}}^{\otimes k-1}_{3}[n])^{3}\right|. $$

The numerator of this Fourier coefficient equals to 1 for any k. Thus, denoting $G(n)=\mathsf {gran}(\widehat {\text {{MAJ}}}^{\otimes k}_{3}[n])$ for n = 3^k we have G(3) = 1 and

$$ G(n)=3G\left( \frac{n}{3}\right)+1. $$

It is straightforward to check that $G(n)=\frac {n-1}{2}$. From this, Lemma 3 and Lemma 14 Theorem 2 follows.