Message Passing Algorithms for MLS-3LIN Problem

Abstract

MLS-3LIN problem is a problem of finding a most likely solution for a given system of perturbed 3LIN-equations under a certain planted solution model. This problem is essentially the same as MAX-3LIN problem. We consider simple and efficient message passing algorithms for this problem, and investigate their success probability, where input instances are generated under the planted solution model with equation probability p and perturbation probability q. For some variant of a typical message passing algorithm, we prove that p=Θ(1/(nlnn)) is the threshold for the algorithm to work w.h.p. for any fixed constant q<1/2.

Appendix: Technical Lemmas

Here we explain spectral lemmas used in this paper. In particular, we give a proof for Lemma 4. Symbols are used in the same way as in the body without explanation.

We begin with stating the lemma in [4] from which Lemma 2 is derived.

Lemma A.1

(Lemma 39 of [4])

Let $\hat{c}_{0}>0$ be any sufficiently large constant. For any $\widetilde{p}$ such that $\hat{c}_{0}/n\le\hat{p}\le(\ln n)^{O(1)}/n$, let D=(d _ij )_1≤i,j≤n be an n×n random symmetric matrix such that d _ii =0 for each 1≤i≤n, and d _ij =1 (resp., d _ij =0) with probability $\hat{p}$ (resp., $1-\hat{p}$) independently for each 1≤i<j≤n. Then for any $\hat{c}_{2}>1$, there exists some $\hat{c}_{1}>0$ such that the following three statements hold w.h.p.

• (1) Let $V''=\{i\in\{1,\ldots,n\}|\sum_{j=1}^{n} d_{ij}>\hat{c}_{2} n\hat{p}\}$, and V′={1,…,n}−V″. Let n′=#V′ and D′=(d _ij ) _i,j∈V′ be the induced n′×n′ matrix. Then we have $n'\ge(1-\mathrm{exp}(-n\hat{p}/\hat{c}_{1}))n$.

• (2) For any unit vector ξ⊥1, we have $\|D'\boldsymbol{\xi}\|\le\hat{c}_{1}\sqrt{n'\hat{p}}$.

• (3) For any W⊆V′ of cardinality $\ge(1-(n\hat{p})^{-10})n$, then D″=(d _ij ) _i,j∈W satisfies $\|D''\boldsymbol{1}-n\hat{p}\boldsymbol{1}\|\le\widehat {c}_{1}n\sqrt{\hat{p}}$.

By checking the proof of [4] as well as that of [1], it is not so hard to confirm that the bound of Lemma A.1 (2) can be shown for D″, which proves (2) of Lemma 2. For the other parts, it is easy to see that Lemma A.1 implies Lemma 2.

Next we apply Lemma 2 to show some properties of matrices B and C. Recall that $\widehat{E}$ and $\widehat{A}_{\widehat{E}}=(\hat{a}_{ij})$ denote respectively the original input instance and the corresponding matrix. We simply write $\widehat{A}$ for $\widehat{A}_{\widehat{E}}$. On the other hand, corresponding to B and C, let $\widehat{B}=(\hat{b}_{ij})$ and $\widehat{C}=(\widehat {c}_{ij})$ denote matrices that have 1 at each ij-entry such that $\hat{a}_{ij}=1$ and $\hat{a}_{ij}=-1$ in $\widehat {A}$ respectively. Note that all $\hat{b}_{ij}$’s for i<j (resp., $\hat{c}_{ij}$’s for i<j) are mutually independent; hence, we can apply Lemma 2 by regarding $\widehat{B}$ (resp., $\widehat{C}$) as matrix D of the lemma. In order to apply Lemma 2, we use $\hat{n}$, $\widehat{B}$ (resp., $\widehat{C}$), and p(1−q) (resp., pq) for n, D, and $\widetilde{p}$ of the lemma. Note that both p(1−q) and pq are larger than $\widetilde{c}_{0}/\hat{n}$ from our choice⁵ of c. Also by applying AlgPre, all ith entries of $\widehat{B}$ (resp., $\widehat{C}$) such that $\sum_{j} \hat{b}_{ij}\ge2\hat{n}p$ (resp., $\sum_{j} \hat{c}_{ij}\ge2\hat{n}p$) are removed from $\widehat{B}$ (resp., $\widehat{C}$). Thus we use p for $\widetilde{p}\,'$. Then the following bounds are obtained by Lemma 2.

Lemma A.2

The following holds w.h.p.

• (1) For any unit vector ξ⊥1, we have $\|B\boldsymbol{\xi}\|\le c_{1}\sqrt{np}$ and $\|C\boldsymbol{\xi}\|\le c_{1}\sqrt{np}$.

• (2) $\|B\overline{\boldsymbol{1}}-np(1-q)\overline{\boldsymbol{1}}\|\le c_{1}\sqrt{np}$ and $\|C\overline{\boldsymbol{1}}-npq\overline{\boldsymbol{1}}\|\le c_{1}\sqrt{np}$.

Now we prove Lemma 4.

Lemma 4

The following holds w.h.p.

• (1) For any unit vector η⊥1 and for any unit vector ξ, we have $|\langle A\boldsymbol{\eta},\boldsymbol{\xi}\rangle|\le 2c_{1}\sqrt{np}$.

• (2) For any λ _i , 2≤i≤n, we have $|\lambda_{i}|\le4c_{1}\sqrt{np}$.

• (3) $\|A\overline{\boldsymbol{1}}-nr\overline{\boldsymbol{1}}\| \le 2c_{1}\sqrt{np}$.

Proof

(1) Let η and ξ be any unit vectors, and we assume that η⊥1. By Cauchy-Schwartz inequality we have |〈B η,ξ〉|≤∥B η∥⋅∥ξ∥. Then by using the above lemma, we have

$$\bigl|\langle B\boldsymbol{\eta},\boldsymbol{\xi}\rangle\bigr| \le \|B\boldsymbol{\eta}\|\cdot\|\boldsymbol{\xi}\| = \|B\boldsymbol{\eta}\| \le c_1\sqrt{np}. $$

Similarly, we have $|\langle C\boldsymbol{\eta},\boldsymbol{\xi}\rangle| \le c_{1}\sqrt{np}$. Thus, we have we have

(2) Let S ₀ be the set of vectors perpendicular to 1. Then since ${\rm dim}(S_{0})=n-1$, by Theorem of Courant-Fischer, we have

$$\lambda_2 \le \max_{\boldsymbol{\eta}\in S_0}\langle A\boldsymbol{\eta },\boldsymbol{\eta}\rangle \le 2c_1\sqrt{np}. $$

On the other hand, to bound λ _n , we express the nth eigenvector as $\boldsymbol{\xi}_{n}=\alpha\overline{\boldsymbol{1}}+\beta \boldsymbol{\eta}$ for some α,β such that α ²+β ²=1, and η is a unit vector perpendicular to $\overline {\boldsymbol{1}}$. Then we have

Here we used the facts that 〈A x,y〉=〈x,A y〉 due to the symmetry of A, and that $\langle A\overline{\boldsymbol{1}},\overline{\boldsymbol {1}}\rangle\ge0$ w.h.p. For the last bound, we used the fact⁶ that 2|α|⋅|β|+β ²<1.62 under the constraint α ²+β ²=1.

(3) The claim follows from the following.

 □