BKW Meets Fourier New Algorithms for LPN with Sparse Parities

Abstract

We consider the Learning Parity with Noise (LPN) problem with sparse secret, where the secret vector $\mathbf {s}$ of dimension n has Hamming weight at most k. We are interested in algorithms with asymptotic improvement in the exponent beyond the state of the art. Prior work in this setting presented algorithms with runtime $n^{c \cdot k}$ for constant $c < 1$, obtaining a constant factor improvement over brute force search, which runs in time ${n \atopwithdelims ()k}$. We obtain the following results:

• We first consider the constant error rate setting, and in this case present a new algorithm that leverages a subroutine from the acclaimed BKW algorithm [Blum, Kalai, Wasserman, J. ACM ’03] as well as techniques from Fourier analysis for p-biased distributions. Our algorithm achieves asymptotic improvement in the exponent compared to prior work, when the sparsity $k = k(n) = \frac{n}{\log ^{1+ 1/c}(n)}$, where $c \in o(\log \log (n))$ and $c \in \omega (1)$. The runtime and sample complexity of this algorithm are approximately the same.

• We next consider the low noise setting, where the error is subconstant. We present a new algorithm in this setting that requires only a polynomial number of samples and achieves asymptotic improvement in the exponent compared to prior work, when the sparsity $k = \frac{1}{\eta } \cdot \frac{\log (n)}{\log (f(n))}$ and noise rate of $\eta \ne 1/2$ and $\eta ^2 = \left( \frac{\log (n)}{n} \cdot f(n)\right)$, for $f(n) \in \omega (1) \cap n^{o(1)}$. To obtain the improvement in sample complexity, we create subsets of samples using the design of Nisan and Wigderson [J. Comput. Syst. Sci. ’94], so that any two subsets have a small intersection, while the number of subsets is large. Each of these subsets is used to generate a single p-biased sample for the Fourier analysis step. We then show that this allows us to bound the covariance of pairs of samples, which is sufficient for the Fourier analysis.

• Finally, we show that our first algorithm extends to the setting where the noise rate is very high $1/2 - o(1)$, and in this case can be used as a subroutine to obtain new algorithms for learning DNFs and Juntas. Our algorithms achieve asymptotic improvement in the exponent for certain regimes. For DNFs of size s with approximation factor $\epsilon$ this regime is when $\log \frac{s}{\epsilon } \in \omega \left( \frac{c}{\log n \log \log c}\right)$, and $\log \frac{s}{\epsilon } \in n^{1 - o(1)}$, for $c \in n^{1 - o(1)}$. For Juntas of k the regime is when $k \in \omega \left( \frac{c}{\log n \log \log c}\right)$, and $k \in n^{1 - o(1)}$, for $c \in n^{1 - o(1)}$.

D. Dachman-Soled—Supported in part by NSF grants #CNS-1933033, #CNS-1453045 (CAREER), and by financial assistance awards 70NANB15H328 and 70NANB19H126 from the U.S. Department of Commerce, National Institute of Standards and Technology.

H. Gong—Most of the work was done while the author was a student at the University of Maryland, College Park.

H. Kippen—Supported in part by the Clark Doctoral Fellowship from the Clark School of Engineering, University of Maryland, College Park.

A Appendix

1.1 A.1 Probability Bounds

The following inequality is used to bound the magnitude of an observed random variable with respect to the true expected value of that random variable. The Chernoff-Hoeffding bound extends the Chernoff bound to random variables with a bounded range. Another important fact is that Chernoff-Hoeffding bound assumes the random variables are independent whereas Chebyshev’s bound applies to arbitrary random variables. The reader in encouraged to refer to [23] for more in depth reading.

Theorem A.1

(Multiplicative Chernoff Bounds).  Let $X_1, X_2, \ldots , X_n$ be n mutually independent random variables. Let $X = \sum _{i=1}^{n}X_i$ and $\mu = \mathbb {E}[X]$,

$$ \Pr [X \le (1-\beta ) \mu ] \le \exp \left( \frac{-\beta ^2 \mu }{2}\right) \text {for all } 0 < \beta \le 1 $$

$$ \Pr [X \ge (1+\beta ) \mu ] \le \exp \left( \frac{-\beta ^2 \mu }{3}\right) \text {for all } 0 < \beta \le 1 $$

Theorem A.2

(Chernoff-Hoeffding).  Consider a set of n independent random variables $X_1, X_2, \ldots , X_n$. If we know $a_i \le X_i \le b_i$, then let $\varDelta _i = b_i - a_i$. Let $X = \sum _{i=1}^{n}X_i$. Then for any $\alpha \in \left( 0,1/2\right)$

$$ \Pr \left( \big | X - \mathbb {E}[X] \big | > \alpha \right) \le 2 \text {exp}\left( \frac{-2\alpha ^2}{\sum _{i=1}^{n}\varDelta _i^2}\right) . $$

Theorem A.3

(Chebyshev’s).  Consider a set of n arbitrary random variable $X_1, X_2, \ldots , X_n$. Let $X = \sum _{i=1}^{n}X_i$. Then for any $\alpha > 0$,

$$ \Pr \left( \big | X - \mathbb {E}[X]\big | \ge \alpha \right) \le \frac{\mathrm {Var}\left[ X\right] }{\alpha ^2}. $$

The following lemma is being used to further simplify the $\mathrm {Var}[X]$ in Theorem A.3.

Lemma A.4

Let $X_1, X_2, \ldots , X_n$ be n arbitrary random variables. Then

$$ \mathrm {Var}\left[ \sum _{i=1}^{n}X_i \right] = \sum _{i=1}^{n}\mathrm {Var}\left[ X_i\right] + 2 \sum _{i=1}^{n}\sum _{j > i}\mathrm {Cov}\left[ X_i, X_j\right] . $$

1.2 A.2 Learning Parities

  In this subsection, we define three Oracles . The first is the standard LPN Oracle, that samples $\mathbf {x}$ uniformly. The second is the noise Oracle, which sets $\mathbf {x}$ to the zero vector. The purpose of this Oracle is to return additional noise sampled identically to the noise found in a normal LPN sample. The third Oracle is the p-biased LPN Oracle, which samples $\mathbf {x}$ according to a p-biased Bernoulli distribution.

Definition A.5

(Bernoulli Distribution). Let $p \in \left[ 0,1\right]$. The discrete probability distribution of a random variable which takes the value 1 with probability $\eta$ and the value 0 with probability $1-\eta$ is called Bernoulli Distribution and it is denoted by $\mathsf {Ber}_\eta$.

Definition A.6

(LPN Oracle). Let secret value $\mathbf {s} \leftarrow {\mathbb {Z}}_2^n$ and let $\eta < 1/2$ be a constant noise parameter. Let $\mathsf {Ber}_\eta$ be a Bernoulli distribution with parameter $\eta$. Define the following distribution $\mathcal {L}^{(1)}_{\mathbf {s}, \eta }$ as follows

$$ \big \{ \left( \mathbf {x}_{}, b_{} \right) |\ \mathbf {x} \leftarrow {\mathbb {Z}}_2^n, \ f_{\mathbf {s}}(\mathbf {x}) := \langle \mathbf {x}, \mathbf {s} \rangle _{}, \ b = f_{\mathbf {s}}(\mathbf {x}) + e,\ e \leftarrow \mathsf {Ber}_\eta \} \in {\mathbb {Z}}_2^{n+1} $$

with the additions being done module 2. Upon calling the LPN Oracle $\mathcal {O^{\mathsf {LPN}}_{ 0 ,\eta }}(\mathbf {s})$, a new sample $\mathsf {s}_{} = \left( \mathbf {x}_{}, b_{} \right)$ from the distribution $\mathcal {L}^{(1)}_{\mathbf {s}, \eta }$ is returned.

Definition A.7

(Noise Oracle). Let secret value $\mathbf {s} \leftarrow {\mathbb {Z}}_2^n$ and let $\eta < 1/2$ be a constant noise parameter. Let $\mathsf {Ber}_\eta$ be a Bernoulli distribution with parameter $\eta$. Define the following distribution $\mathcal {L}^{(2)}_{\mathbf {s}, \eta }$ as follows

$$ \big \{ \left( \mathbf {x}_{}, b_{} \right) |\ \mathbf {x} := 0^n, \ f_{\mathbf {s}}(\mathbf {x}) := \langle \mathbf {x}, \mathbf {s} \rangle _{}, \ b = f_{\mathbf {s}}(\mathbf {x}) + e,\ e \leftarrow \mathsf {Ber}_\eta \} \in {\mathbb {Z}}_2^{n+1} $$

with the additions being done module 2. Upon calling the Noise Oracle $\mathcal {\tilde{O}}_{\eta }$ a new sample $\mathsf {s}_{} = \left( \mathbf {x}_{}, b_{} \right)$ from the distribution $\mathcal {L}^{(2)}_{\mathbf {s}, \eta }$ is returned.

Definition A.8

(p-biased LPN Oracle). Let secret value $\mathbf {s} \leftarrow {\mathbb {Z}}_2^n$ and let $\eta < 1/2$ be a constant noise parameter. Let $\mathsf {Ber}_\eta$ be a Bernoulli distribution with parameter $\eta$ and $\mathsf {Ber}_{(1-p)/2}^n$ be Bernoulli distribution with parameter $(1-p)/2$ over n coordinates. Define the following distribution $\mathcal {L}^{(3)}_{\mathbf {s}, \eta , p}$ as follows

$$ \big \{ \left( \mathbf {x}_{}, b_{} \right) |\ \mathbf {x} \leftarrow \mathsf {Ber}_{(1-p)/2}^n,\ f_{\mathbf {s}}(\mathbf {x}) := \langle \mathbf {x}, \mathbf {s} \rangle _{}, \ b = f_{\mathbf {s}}(\mathbf {x}) + e,\ e \leftarrow \mathsf {Ber}_\eta \} \in {\mathbb {Z}}_2^{n+1} $$

with the additions being done modulo 2. Upon calling the p-biased LPN Oracle $\mathcal {O^{\mathsf {LPN}}_{ p ,\eta }}(\mathbf {s})$ a new sample $\mathsf {s}^{p}_{} = \left( \mathbf {x}_{}, b_{} \right)$ from the distribution $\mathcal {L}^{(3)}_{\mathbf {s}, \eta ,p}$ is returned.

As our algorithms require linear combinations of LPN samples, we present the following lemma that describes the noise growth associated with the linear combination.

Lemma A.9

(New Sample Error [5]).  Given a set of $\ell$ samples $(\mathbf {x}_1, b_1), \ldots , (\mathbf {x}_\ell , b_\ell )$ from an LPN Oracle $\mathcal {O^{\mathsf {LPN}}_{ 0 ,\eta }}(\mathbf {s})$ with secret $\mathbf {s}$, where the choice of samples may depend on the values of $\mathbf {x}_i$ but not on the values of $b_i$, then the new sample $\mathsf {s}_{\ell +1}$ can be formed as follows $\mathsf {s}_{\ell +1} = \sum _{i=1}^{\ell } \mathsf {s}_{i}$ which has the property that $b_{\ell +1}$ is independent of $\mathbf {x}_{\ell +1}$ and the probability that the label of the constructed sample is correct is as follows: $(1-\eta ') = \Pr [b' = \langle \mathbf {x}_{\ell +1}, s \rangle ] = \frac{1}{2} + \frac{1}{2}(1-2\eta )^\ell$.

For reference we additionally provide the runtime of the original $\mathsf {BKW}$ algorithm:

Theorem A.10

($\mathsf {BKW}$ [5]).  The length-n parity problem, for noise rate $\eta$ for any constant less than 1/2, can be solved with number of samples and total computation time of $2^{O(n/\log n)}$.

For sample i, the j-th coordinate of $\mathbf {x}$ is denoted by $\mathsf {s}_{i}.\mathbf {x}[j]$ and the j-th coordinate of $\mathbf {s}$ is denoted by $\mathsf {s}_{i}.\mathbf {s}[j]$. For simplicity, given two sample pairs $\mathsf {s}_{1} = \left( \mathbf {x}_{1}, b_{1} \right)$ and $\mathsf {s}_{2} = \left( \mathbf {x}_{2}, b_{2} \right)$ a new sample $\mathsf {s}_{3} = \mathsf {s}_{1} + \mathsf {s}_{2}$ can be formed by $\mathsf {s}_{3} = \left( \mathbf {x}_1 + \mathbf {x}_2, b_1 + b_2\right)$ with the additions being done $\text {mod}\ 2$.

1.3 A.3 Miscellaneous

Definition A.11

(Restricted Left Kernel).  Given a matrix $\mathbf {A} \in {\mathbb {Z}}_2^{m \times n}$ for $m \le n$ and set $R \subset [n]$ such that $|R| < m$, $\mathsf {RLK}$ first finds a vector $\mathbf {u} \in {\mathbb {Z}}_2^m$ such that $\mathbf {v} = \mathbf {u} \cdot \mathbf {A}$ and $\mathbf {v}|_{R} = 0^{|R|}$. The $\mathsf {RLK}$ algorithm returns $(\mathbf {v}, \mathbf {u}) := \mathsf {RLK}(\mathbf {A}, R)$.

Note that the $\mathsf {RLK}$ algorithm mentioned above can be implemented by simply modifying matrix $\mathbf {A}$ and only takes the columns pointed by set R, i.e. restriction of $\mathbf {A}$ to only columns pointed by R. Let’s denote the new matrix by $\mathbf {A}'$, find a vector in left kernel of $\mathbf {A}'$ and call it $\mathbf {u}$. Then $\mathbf {v}$ can simply be computed as $\mathbf {v} = \mathbf {u} \cdot \mathbf {A}$.

Definition A.12

(Hamming Weight). Given a vector $\mathbf {u} \in {\mathbb {Z}}_2^m$, $\mathsf {weight}(\mathbf {u})$ returns the number of 1’s in vector $\mathbf {u}$, i.e. the Hamming weight of $\mathbf {u}$.

1.4 A.4 Proof of Lemma 3.1

We first show that each coordinate of $\mathbf {x}'$ is set to 0 with independent probability $(1+p)/2$. The probability that a coordinate j of $\mathbf {x}'$ in sample $\mathsf {s}^{p}_{}$ is set to 0 after running $\mathsf {BKW}_{\mathsf {R}}$ can be computed as follows:

$$\begin{aligned} \Pr \left[ \mathbf {x'}[j] = 0\right]&= \Pr \left[ \mathbf {x'}[j] = 0\ |\ j \in R\right] \cdot \Pr \left[ j \in R\right] + \Pr \left[ \mathbf {x'}[j] = 0\ |\ j \notin R\right] \cdot \Pr \left[ j \notin R\right] \\&= 1 \cdot p + 1/2 \cdot (1 - p) = (1+p)/2 \end{aligned}$$

To show that the label $b'$ is correct with probability $\eta '$ and that the correctness of the label is independent of the instance $\mathbf {x'}, \mathbf {s}$, note that $\mathbf {x'}$ is always constructed by XOR’ing a set of exactly $2^\mathfrak {a}$ number of samples and that the choice of the set of XOR’ed samples depends only on the random coins of the algorithm and on the $\mathbf {x}$ values, which are independent of the e value. Therefore, we can apply Lemma A.9 to conclude that the noise is independent and that $b'$ is correct with probability $\eta ' = \frac{1}{2} - \frac{1}{2} (1 - 2 \eta )^{\sqrt{2np}}$.

1.5 A.5 Proof of Theorem 3.2

From the description of $\mathsf {BKW}_{\mathsf {R}}$, it is clear to see that it takes $O(\mathfrak {a}2^{\mathfrak {b}})$ LPN samples and running time to generate a p-biased sample, where $\mathfrak {a} = \log (2np)/2, \mathfrak {b} = \lceil |R|/\mathfrak {a}\rceil$. Remember that the $\mathsf {BKW}_{\mathsf {R}}$ algorithm will abort if $|R| \ge 2pn$ or $|R| \le pn/2$, i.e. $\mathsf {Event 1}$ occurs. By showing that $\mathsf {Event 1}$ occurs with probability at most $2 \exp (-p \cdot n/8)$ , we obtain that $\mathsf {BKW}_{\mathsf {R}}$ runs in time $O(2^{\frac{4np}{\log (2np)} }\cdot \log (2np))$ with probability at least $1 - 2 \exp (-p \cdot n/8)$.

To bound the probability of $\mathsf {Event 1}$ occurring, we notice that by multiplicative Chernoff bounds in Theorem A.1, we can bound the size of set R as follows:

$$\begin{aligned} \Pr \left[ |R| \ge 2pn \right]&\le \exp (-p \cdot n/3)\\ \Pr \left[ |R| \le pn/2 \right]&\le \exp (-p \cdot n/8)\\ \Pr \left[ |R| \ge 2pn \vee |R| \le pn/2 \right]&\le \exp (-p \cdot n/3) + \exp (-p \cdot n/8) \le 2 \exp (-p \cdot n/8)\\ \Pr \left[ pn/2< |R| < 2pn \right]&> 1 - 2 \exp (-p \cdot n/8) \\ \end{aligned}$$

1.6 A.6 Proof of Lemma 3.3

Before proving Lemma 3.3, we present the following simple claims about the number of samples needed to estimate the Fourier Coefficient of a single index. Based on Claim 2.2, the magnitude of Fourier coefficient of the indexes with secret value of 0 is equal to 0, while for the secret coordinates 1 that is equal to $\varepsilon = (1 - 2\eta ') \cdot p^{k-1}\sqrt{1-p^2}$. In the Following Claim we compute how many samples are needed to estimate the magnitude of Fourier coefficient within distance of $\varepsilon /2$ of correct value. We will bound the failure probability with $\delta /n$.

Claim A.13

  For every $j \in [n]$, $\hat{b}_{p}(\{j\}) = \mathbb {E}[b\cdot \chi _{\{j\}, p}(\mathbf {x}))]$, where $(\mathbf {x}, b) \sim \mathcal {O^{\mathsf {LPN}}_{ p ,\eta '}}(\mathbf {s})$, can be estimated within additive accuracy $\frac{\varepsilon }{2}$ and confidence $1-\frac{\delta }{n}$ using $\frac{8}{\varepsilon ^2} \cdot \frac{1+p}{1-p} \cdot \ln (2n/ \delta )$ number of samples.

Proof

The estimate of $\hat{b}_{p}(\{j\})$ based on the m samples $\mathsf {s}^{p}_{i} = \left( \mathbf {x}_{i}, b_{i} \right)$ is.

$$ \hat{b}_{\text {estimate}}(\{j\}) = \frac{1}{m} \sum _{i = 1}^{m} b_i \cdot \chi _{\{j\}, p}(\mathbf {x}_i) $$

and notice that $\mathbb {E}\left[ \hat{b}_{\text {estimate}}(\{j\})\right] = \hat{b}_{p}(\{j\})$. Lets denote $X_i = \frac{1}{m} \cdot b_i \cdot \chi _{\{j\}, p}(\mathbf {x}_i)$, then note that $|X_i| \le (1/m)\sqrt{\frac{1+p}{1-p}}$. Finally by Chernoff-Hoeffding bound of Theorem A.2 we have the following.

$$ \Pr \left[ \left| \hat{b}_{\text {estimate}}(\{j\}) - \hat{b}_{p}\left( \{j\}\right) \right| \ge \varepsilon /2 \right] \le 2\ \text {exp}\left( \frac{-m \varepsilon ^2}{8} \cdot \frac{1-p}{1+p}\right) $$

Bounding the right hand side by $\delta /n$ and solving for m gives the desired value for number of samples.

Proof

(Proof of Lemma 3.3). Invoking Claim 2.2, we have that for j such that $\mathbf {s}[j]=1$ $\hat{b}_p(\{j\}) = (1 - 2\eta ') \cdot p^{k-1}\sqrt{1-p^2}$ while for j such that $\mathbf {s}[j]=0$ , $\hat{b}_p(\{j\}) = 0$. It is clear by inspection that Algorithm 2 succeeds when it correctly estimates the values of $\hat{b}_p(\{j\})$ to within additive $\varepsilon /2 := (1 - 2\eta ') \cdot p^{k-1}\sqrt{1-p^2}/2$ for all $j \in [n]$. By Claim A.13, $\frac{8}{\varepsilon ^2} \cdot \frac{1+p}{1-p} \cdot \ln (2n/ \delta )$ number of samples are sufficient to estimate a single coordinate within additive $\varepsilon /2$ of its correct value with confidence $1- \frac{\delta }{n}$. By a union bound, the success probability of estimating all coordinates to within additive $\varepsilon /2$ is $1- \delta$.    $\Box$

1.7 A.7 Proof of Lemma 4.1

The proof is similar to the proof of Lemma 3.1 and noticing that the $\mathsf {SamP}$ algorithm uses $2np+1$ samples to generate a single p-biased sample. Two p-biased samples $\mathbf {x}'_i, \mathbf {x}'_j$, $j > i$ are pairwise independent, unless the same linear combination of samples in $\mathcal {S}$ was used to generate both of them. But in that case, during execution, the condition $\mathbf {x}'_j|_{R_i} = 0^{|R_i|}$ would evaluate to true, which means that $\mathsf {Event 2}$ occurred and so fresh samples (not from $\mathcal {S}$) would be used to generate $\mathbf {x}'_j$.

In the rest of the proof we switch to the $\pm 1$ representation instead of the Boolean representation. The sample $\mathsf {s}^{p}_{i} = \left( \mathbf {x}'_{i}, b'_{i} \right)$ is obtained from the samples in set $\mathcal {O}_i$ alongside some extra error samples from Noise Oracle $\mathcal {\tilde{O}}_{\eta }$. In the following proof these are denoted by $e_1, e_2, \ldots , e_{2np+1}$. Moreover, notice that the sample $\mathsf {s}^{p}_{j} = \left( \mathbf {x}'_{j}, b'_{j} \right)$, obtained from set $\mathcal {O}_j$, has at most t elements in common with the sample obtained from the set $\mathcal {O}_i$. Hence we can represent the error in sample $\mathsf {s}^{p}_{j} = \left( \mathbf {x}'_{j}, b'_{j} \right)$ as $e_1, e_2, \ldots , e_t, e''_{t+1} \ldots e''_{2np+1}$. For the ease of notation we assumed that the t samples which are in common are at index 1 to t.

$$\begin{aligned} \mathrm {Cov}[e'_i, e'_j]&= \mathrm {Cov}[e_1 \cdot e_2 \ldots e_t \cdot e_{t+1} \ldots e_{2np+1}\ ,\ e_1 \cdot e_2 \ldots e_t \cdot e''_{t+1} \ldots e''_{2np+1}]\\&= \mathbb {E}[e_1^{2} \cdot e_2^{2} \ldots e_t^{2} \cdot e_{t+1} \ldots e_{2np+1} \cdot e''_{t+1} \ldots e''_{2np+1} ]\\&\qquad - \mathbb {E}[e_1 \cdot e_2 \ldots e_{2np+1}]\ \mathbb {E}[e_1 \cdot e_2 \ldots e_t \ldots e''_{t+1} \ldots e''_{2np+1}]\\&= \left( 1-2\eta \right) ^{2(2np-t)+2} - \left( 1-2\eta \right) ^{4np+2} \end{aligned}$$

Where the last line follows from the independence of errors, $\mathbb {E}[e_i] = 1-2\eta$ and $\mathbb {E}[e_i^{2}] = 1$.

1.8 A.8 Proof of Theorem 4.2

Assuming $\mathsf {Event 1}$ and $\mathsf {Event 2}$ do not occur, the sample complexity and runtime can be verified by inspection and assuming $\mathsf {RLK}$ takes $\mathsf {poly}(np)$ time.

It remains to bound the probability of $\mathsf {Event 1}$ and $\mathsf {Event 2}$. We can upper bound the probability of $\mathsf {Event 1}$ by $2 \exp (-p \cdot n/8)$, as in the proof of Theorem 3.2.

To upperbound the probability of $\mathsf {Event 2}$, we note that assuming $\mathsf {Event 1}$ does not occur, $\mathsf {Event 2}$ occurs only if one of the following two events occur:

• $\mathsf {Event' 1}$: For some distinct $i,j \in \mathsf {maxnum}$, $|R_i \cap R_j| \ge np/4$.

• $\mathsf {Event' 2}$: For some distinct $i,j \in \mathsf {maxnum}$, $|R_i \setminus R_j| \ge np/4$ and $\mathbf {x}'_j|_{R_i \setminus R_j} = 0^{|R_i \setminus R_j|}$.

Since for distinct i, j, each coordinate $\ell \in [n]$ is placed in both $R_i$ and $R_j$ with probability $p^2$, by a union bound over all pairs i, j and a standard Chernoff bound, $\mathsf {Event' 1}$ can be upperbounded by:

$$ \mathsf {maxnum}^2 \cdot \exp (-n/48) = (np)^t \cdot \exp (-n/48). $$

Since for any $\mathbf {x}'_j$, the coordinates outside of $R_j$ are uniformly random, $\mathsf {Event' 2}$ can be upperbounded by:

$$ \mathsf {maxnum}^2 \cdot {1/2}^{np/4} = (np)^t \cdot {1/2}^{np/4}. $$

1.9 A.9 Proof of Lemma 4.3

Similar to Subsect. 3.2, before proving Lemma 4.3, we first present the following claim about the number of samples needed to estimate the Fourier Coefficient of a single index. The algorithm gets access to $8\log (n)$ sets of p-biased samples. In the following claim we first prove how many samples are needed to be able to approximate the Fourier coefficient within additive distance of $\epsilon /2$ and later discuss how by repeating the approximation step, i.e. step 2b in Fig. 4, will reduce the error in approximation even further.

Claim A.14

For $\delta \in [0,1]$, $p \in (0,1)$, given $8\log (n)$ independent sets of samples $\mathcal {S}_1, \mathcal {S}_2, \ldots , \mathcal {S}_{8\log (n)}$ that each of size $\mathsf {num}:=O \left( \frac{1}{(1-2\eta )^{4np+2}p^{2(k-1)}(1-p^2)}\right)$ and each satisfying the properties given in Lemma 4.1 for some $t \in \varTheta (1/\eta )$, then for every $j \in [n]$, $\hat{b}_{p}(\{j\}) = \mathbb {E}[b\cdot \chi _{\{j\}, p}(\mathbf {x}))]$ can be estimated within additive accuracy $\frac{\epsilon }{2} = (1 - 2\eta ')p^{k-1}\sqrt{1-p^2}/2$ for $\eta ' = \frac{1}{2} - \frac{1}{2} (1 - 2 \eta )^{2np+1}$ with confidence $1-\frac{\delta }{n}$.

Proof

Let $X = \frac{1}{m} \sum _{i = 1}^{m} b_i \cdot \chi _{S, p}(\mathbf {x}_i)$. Let f be a parity function. Assuming $S = \{k\}$, let $X_i = \frac{1}{m} \cdot b_i \cdot \chi _{\{k\}, p}(\mathbf {x}_i)$. First we compute $\mathrm {Cov}[X_i, X_j]$ for k such that $\mathbf {s}[k] = 1$

$$\begin{aligned} \mathrm {Cov}[X_i, X_j]&= \mathrm {Cov}\left[ \frac{1}{m} \cdot b'_i \cdot \chi _{\{k\}, p}(\mathbf {x}'_i)\ ,\ \frac{1}{m} \cdot b'_j \cdot \chi _{\{k\}, p}(\mathbf {x}'_j)\right] \nonumber \\ \mathrm {Cov}[X_i, X_j]&= \frac{1}{m^2} \cdot \mathrm {Cov}\left[ b'_i \cdot \chi _{\{k\}, p}(\mathbf {x}'_i)\ ,\ b'_j \cdot \chi _{\{k\}, p}(\mathbf {x}'_j)\right] \nonumber \\&= \frac{1}{m^2} \cdot \mathrm {Cov}\left[ \left( \prod _{u : \mathbf {s}[u] = 1} \mathbf {x}'_i[u]\right) \cdot e'_i \cdot \frac{\mathbf {x}'_i[k]-p}{\sqrt{1-p^2}}\ ,\ \left( \prod _{v : \mathbf {s}[v] = 1} \mathbf {x}'_j[v]\right) \cdot e'_j \cdot \frac{\mathbf {x}'_j[k]-p}{\sqrt{1-p^2}} \right] \end{aligned}$$

(A.1)

$$\begin{aligned}&= \frac{1}{m^2} \cdot \frac{1}{1-p^2} \Biggl ( \mathrm {Cov}\left[ \left( \prod _{u : \mathbf {s}[u] = 1 \wedge u \ne k} \mathbf {x}'_i[u]\right) \cdot e'_i\ ,\ \left( \prod _{v : \mathbf {s}[v] = 1 \wedge v \ne k } \mathbf {x}'_j[v]\right) \cdot e'_j\right] - \nonumber \\&\ \quad \mathrm {Cov}\left[ \left( \prod _{u : \mathbf {s}[u] = 1 \wedge u \ne k} \mathbf {x}'_i[u]\right) \cdot e'_i\ ,\ p \cdot \left( \prod _{v : \mathbf {s}[v] = 1} \mathbf {x}'_j[v]\right) \cdot e'_j\right] - \nonumber \\&\ \quad \mathrm {Cov}\left[ p \cdot \left( \prod _{u : \mathbf {s}[u] = 1} \mathbf {x}'_i[u]\right) \cdot e'_i\ ,\ \left( \prod _{v : \mathbf {s}[v] = 1 \wedge v \ne k} \mathbf {x}'_j[v]\right) \cdot e'_j\right] + \nonumber \\&\ \quad \mathrm {Cov}\left[ p \cdot \left( \prod _{u : \mathbf {s}[u] = 1} \mathbf {x}'_i[u]\right) \cdot e'_i\ ,\ p \cdot \left( \prod _{v : \mathbf {s}[v] = 1} \mathbf {x}'_j[v]\right) \cdot e'_j\right] \Biggl ) \end{aligned}$$

(A.2)

$$\begin{aligned}&= \frac{1}{m^2} \cdot \frac{1}{(1-p^2)} \left( p^{2(k-1)} \mathrm {Cov}\left[ e'_i,e'_j\right] - 2p^{2k} \mathrm {Cov}\left[ e'_i,e'_j\right] + p^{2(k+1)} \mathrm {Cov}\left[ e'_i,e'_j\right] \right) \end{aligned}$$

(A.3)

$$\begin{aligned}&= m^{-2} p^{2(k-1)} (1-p^2) \mathrm {Cov}\left[ e'_i,e'_j\right] \nonumber \\&= m^{-2} p^{2(k-1)} (1-p^2) \left[ \left( 1-2\eta \right) ^{2(2np-t)+2} - \left( 1-2\eta \right) ^{4np+2} \right] \end{aligned}$$

(A.4)

where Eq. (A.1) follows from definition of Fourier Coefficients and noting that $b'_{i}$ is multiplications of $\mathbf {x}_i$s and error term $e_i$, Eq. (A.2) follows from properties of Covariance, Eq. (A.3) follows from independence of $\mathbf {x}'_i$s and Eq. (A.4) follows from Lemma 4.1. We can also bound $\mathrm {Var}[X_i]$ as follows

$$\begin{aligned} \mathrm {Var}[X_i]&= \mathrm {Var}\left[ \frac{1}{m} \cdot b'_i \cdot \chi _{\{k\}, p}(\mathbf {x}'_i) \right] \nonumber \\&= \frac{1}{m^2} \cdot \mathrm {Var}\left[ \left( \prod _{u : \mathbf {s}[u] = 1} \mathbf {x}'_i[u]\right) \cdot e'_i \cdot \frac{\mathbf {x}'_i[k]-p}{\sqrt{1-p^2}} \right] \nonumber \\&= \frac{1}{m^2} \cdot \frac{1}{1-p^2}\left( \mathrm {Var}\left[ \left( \prod _{u : \mathbf {s}[u] = 1 \wedge u \ne k} \mathbf {x}'_i[u]\right) \cdot e'_i \right] - p^2 \cdot \mathrm {Var}\left[ \left( \prod _{v : \mathbf {s}[v] = 1} \mathbf {x}'_i[u]\right) \cdot e'_i \right] \right) \nonumber \\&= \frac{1}{m^2} \cdot \frac{1}{1-p^2}\Biggl ( \mathbb {E}\left[ \left( \prod _{u : \mathbf {s}[u] = 1 \wedge u \ne k} \mathbf {x}^{'2}_i[u]\right) \cdot e^{'2}_i \right] - \mathbb {E}\left[ \left( \prod _{u : \mathbf {s}[u] = 1 \wedge u \ne k} \mathbf {x}'_i[u]\right) \cdot e'_i \right] ^2 - \nonumber \\&\ \quad p^2 \cdot \mathbb {E}\left[ \left( \prod _{u : \mathbf {s}[u] = 1} \mathbf {x}^{'2}_i[u]\right) \cdot e^{'2}_i \right] + p^2 \cdot \mathbb {E}\left[ \left( \prod _{u : \mathbf {s}[u] = 1 } \mathbf {x}'_i[u]\right) \cdot e'_i \right] ^2 \Biggl ) \end{aligned}$$

(A.5)

$$\begin{aligned}&= \frac{1}{m^2} \cdot \frac{1}{1-p^2} \left( 1-p^{2(k-1)}(1-2\eta )^{2np} - p^2 + p^{2(k+1)}(1-2\eta )^{2np}\right) \end{aligned}$$

(A.6)

$$\begin{aligned} =m^{-2} \bigg (1 - p^{2(k-1)}(1+p^2)(1-2\eta )^{2np} \bigg ) \le m^{-2} \end{aligned}$$

where Eq. (A.5) follows from properties of variance and Eq. (A.6) follows from independence of $\mathbf {x}'_i$s. Then we have the following bound from Chebyshev’s bound of Theorem A.3

$$\begin{aligned} \Pr \left[ |X - \mathbb {E}[X]| \ge \varepsilon /2\right]&\le \frac{\sum _{i=1}^{m}\mathrm {Var}\left[ X_i\right] + 2 \sum _{i=1}^{m}\sum _{j > i}\mathrm {Cov}\left[ X_i, X_j\right] }{\varepsilon ^2/4}\\&\le 4 \cdot \frac{m^{-1} + p^{2(k-1)} (1-p^2) \left[ (1-2\eta )^{2(2np-t)+2}-(1-2\eta )^{4np+2}\right] }{\varepsilon ^2} \end{aligned}$$

By substituting $\varepsilon = (1 - 2\eta ') \cdot p^{k-1}\sqrt{1-p^2}$ for $\eta ' = \frac{1}{2} - \frac{1}{2} (1 - 2 \eta )^{2np+1}$, we can bound the right hand side by a constant less than 1/2 by setting $t < -\frac{\ln (9/8 - 1/c)}{2\ln (1-2\eta )}$ and setting $m = c\cdot \frac{1}{(1-2\eta )^{4np+2}p^{2(k-1)}(1-p^2)}$, where $c > 8$. We use random variable $Y_{i'}$ to represents whether the value of $\mathsf {count}$ in step $i'$ is increased or not, specifically $Y_{i'} = 1$ represents the event that $\mathsf {count}$ is increased in step $i'$. Assume we repeat the protocol for T rounds in total. Let $Y = (1/T) \cdot \sum _{i'=1}^{T} Y_{i'}$. First, take the case that j such that $\mathbf {s}[j] = 0$ , we know that in each step of loop over $i'$, $\Pr [Y_{i'} = 1] = 1/2 - \epsilon$. Note that the algorithm is run T times using independent sets $\mathcal {S}_{i'}$ each time and index j is only added if in the majority of the runs its estimated Fourier coefficient is more than $\varepsilon / 2$. Using Chernoff bound, we can bound $\Pr [Y \ge T/2] \le 1/n$.

$$\begin{aligned} \Pr [\text {index { j} is added to set } \mathcal {S'}]&= \Pr [\mathsf {count} \ge T / 2] \\&= \Pr [\frac{\sum _{i'=1}^{T} Y_{i'}}{T} \ge \frac{1}{2}] \\&\le \Pr \left[ |Y - E[Y]| > \varepsilon \right] \le 2 \exp (-2T\varepsilon ^2) \end{aligned}$$

We can bound the right hand side by $\frac{\delta }{n}$ for constant $\delta$ by setting $T = 8 \log (n)$ and $\varepsilon = 1/4$. Similar argument applies to the case for j such that $\mathbf {s}[j] = 1$.    $\Box$

Proof (Proof of Lemma 4.3)

Invoking Claim 2.2, we have that for j such that $\mathbf {s}[j] = 1$ $\hat{b}_p(\{j\}) = (1 - 2\eta ') \cdot p^{k-1}\sqrt{1-p^2}$ while for j such that $\mathbf {s}[j] = 0$, $\hat{b}_p(\{j\}) = 0$. It is clear by inspection that Algorithm in Fig. 4 succeeds when it correctly estimates the values of $\hat{b}_p(\{j\})$ to within additive $\varepsilon /2 := (1 - 2\eta ') \cdot p^{k-1}\sqrt{1-p^2}/2$ for all $j \in [n]$. By Claim A.14, we need $8\log (n)$ sets such that each set has $O\left( \frac{1}{(1-2\eta )^{2np+2}p^{2(k-1)}(1-p^2)}\right)$ number of p-biased samples. So in total $\mathsf {num}\cdot 8\log (n) = O\left( \frac{1}{(1-2\eta )^{2np+2}p^{2(k-1)}(1-p^2)} \cdot \log (n)\right)$ number of p-biased samples are sufficient to estimate a single coordinate within additive $\varepsilon /2$ of its correct value with confidence $1- \frac{\delta }{n}$. By a union bound, the success probability of estimating all coordinates to within additive $\varepsilon /2$ is $1- \delta$.    $\Box$