Exact Low-Rank Matrix Completion from Sparsely Corrupted Entries Via Adaptive Outlier Pursuit

Abstract

Recovering a low-rank matrix from some of its linear measurements is a popular problem in many areas of science and engineering. One special case of it is the matrix completion problem, where we need to reconstruct a low-rank matrix from incomplete samples of its entries. A lot of efficient algorithms have been proposed to solve this problem and they perform well when Gaussian noise with a small variance is added to the given data. But they can not deal with the sparse random-valued noise in the measurements. In this paper, we propose a robust method for recovering the low-rank matrix with adaptive outlier pursuit when part of the measurements are damaged by outliers. This method will detect the positions where the data is completely ruined and recover the matrix using correct measurements. Numerical experiments show the accuracy of noise detection and high performance of matrix completion for our algorithms compared with other algorithms.

Appendix

This appendix provides the mathematical proofs of the theoretical results in Sect. 4.

1.1 Proof of Theorem 1

Proof

When we overestimate $K$, the $K$ outliers will be found to make the objective function 0. Therefore, we only need to consider the $(p-K)$ correct entries. In the mean time, some non-outliers (the overestimated $\varDelta K$ entries) are also considered as outliers and will not be used for MC. As we know, if the number of given entries in one row (or column) is less than $r$, the reconstructed matrix is not unique. Since $\varDelta K$ of the $(p-K)$ known entries will not be used in reconstructing the matrix, when $\varDelta K$ is large enough to make the number of known entries in one row (or column) less than $r$, we will have more than one solution. It is easily seen that the smallest number of known entries in one row (or column) of the matrix is less than or equal to $\lfloor (p-K)/m\rfloor$ (or $\lfloor (p-K)/n\rfloor$), here $\lfloor x\rfloor$ is the largest integer that does not exceed $x$. Without loss of generality, let us assume column $j$ has the smallest number of known entries. It is obvious that this number is bounded by $\min (\lfloor (p-K)/m\rfloor , \lfloor (p-K)/n\rfloor )$. To make the number of known entries in this column no less than $r$, the smallest number of entries to be deleted should not exceed $\min (\lfloor (p-K)/m\rfloor , \lfloor (p-K)/n\rfloor )-r+1$. Thus if $\varDelta K$ is greater than $\min (\lfloor (p-K)/m\rfloor , \lfloor (p-K)/n\rfloor )-r$, the reconstructed matrix will not be unique.

1.2 Proof of Theorem 2

Proof

Since the probability that a certain location is chosen is fixed and equals $q\!=\!(p-K)/ (mn)$. In addition, whether one entry is chosen or not is independent of other entries. The number of known entries in each row (or column) of this matrix follows binomial distribution. For each row, the cumulative distribution can be expressed as

$$\begin{aligned} F(x,n,q)=P(X\le x)=\sum _{i=0}^{\lfloor x \rfloor }\left(\begin{array}{c}n\\ i\end{array}\right)q^i(1-q)^{n-i}, \end{aligned}$$

(7.1)

where $X$ is the number of known entries in this row.

From the Hoeffding’s inequality, we have

$$\begin{aligned} F(k,n,q)\le \exp \left(-2{(nq-k)^2\over n}\right), \end{aligned}$$

(7.2)

for any integer $k\le nq$. Since the distribution of the number of known entries in each row is independent, we can find the upper bound for the probability that there exists one row with at most $k$ given entries:

$$\begin{aligned} P(\min (k_1^r,k_2^r,\ldots ,k_m^r)\le k)\le 1-\left(1-\exp \left(-2{(nq-k)^2\over n}\right)\right)^m:=P_1. \end{aligned}$$

(7.3)

Here $k_i^r$ stands for the number of given entries in the $i^{th}$ row. Similarly the upper bound for the probability that there exists one column with at most $k$ given entries can be expressed as follows:

$$\begin{aligned} P(\min (k_1^c,k_2^c,\ldots ,k_n^c)\le k)\le 1-\left(1-\exp \left(-2{(mq-k)^2\over m}\right)\right)^n:=P_2. \end{aligned}$$

(7.4)

where $k_j^c$ is defined as the number of given entries in the $j$ th column. Combing these two together, we have

$$\begin{aligned} P(\min (k_1^r,k_2^r,\ldots ,k_m^r,k_1^c,k_2^c,\ldots ,k_n^c)\le k)\le P_1+P_2\le 2\max (P_1,P_2), \end{aligned}$$

(7.5)

which means the probability that there exists one row or column with at most $k$ given entries can be bounded by $2\max (P_1,P_2)$.

Let us first assume $P_1>P_2$. Defining

$$\begin{aligned} P_0=2\left(1-\left(1-\exp \left(-2{(nq-k)^2\over n}\right)\right)^m\right), \end{aligned}$$

(7.6)

we then have

$$\begin{aligned} \exp \left(-2{(nq-k)^2\over n}\right)=1-\left(1-{P_0\over 2}\right)^{1/m} \end{aligned}$$

(7.7)

$$\begin{aligned} \Longrightarrow \ k=nq-\sqrt{{-n\over 2}\log (1-(1-P_0/2)^{1/m})}. \end{aligned}$$

(7.8)

When $P_1<P_2$, we have

$$\begin{aligned} k=mq-\sqrt{{-m\over 2}\log (1-(1-P_0/2)^{1/n})}. \end{aligned}$$

(7.9)

We define $K_1$ as the minimal of these two values. Hence given $P_0$, the probability of having one row or column with at most $K_1$ entries is less than $P_0$.

Besides the Hoeffding’s inequality, we also have Chernoff’s inequality,

$$\begin{aligned} F(k,n,q)\le \exp \left(-{1\over 2q}{(nq-k)^2\over n}\right). \end{aligned}$$

(7.10)

In this case

$$\begin{aligned} P_1&=1-\left(1-\exp \left(-{1\over 2p}{(nq-k)^2\over n}\right)\right)^m\end{aligned}$$

(7.11)

$$\begin{aligned} P_2&=1-\left(1-\exp \left(-{1\over 2p}{(mq-k)^2\over m}\right)\right)^n. \end{aligned}$$

(7.12)

After similar calculation, we have

$$\begin{aligned} k&=nq-\sqrt{{-2nq}\log (1-(1-P_0/2)^{1/m})} \end{aligned}$$

(7.13)

for $P_1>P_2$, and

$$\begin{aligned} k&=mq-\sqrt{{-2mq}\log (1-(1-P_0/2)^{1/n})} \end{aligned}$$

(7.14)

when $P_1<P_2$. Similarly we define $K_2$ to be the smaller one of these two values, and the probability of having one row or column with at most $K_2$ entries is less than $P_0$. Combining the results from two inequalities together, we know that with at most $P_0$ probability there exists one row or column with at most $\max (K_1,K_2)$ given entries.