Abstract:
In many application areas ranging from bioinformatics to imaging, we are faced with the following question: can we recover a sparse vector xo ∈ ℝN from its undersampled s...Show MoreMetadata
Abstract:
In many application areas ranging from bioinformatics to imaging, we are faced with the following question: can we recover a sparse vector xo ∈ ℝN from its undersampled set of noisy observations y ∈ ℝn, y = Axo+w. The last decade has witnessed a surge of algorithms and theoretical results to address this question. One of the most popular schemes is the ℓp-regularized least squares given by the following formulation:x̂(y, p) ∈ arg minx (1/2)∥y - Ax∥22 + γ∥x∥pp, where p ∈ [0, 1]. Among these optimization problems, the case p = 1, also known as LASSO, is the best accepted in practice, for the following two reasons. First, thanks to the extensive studies performed in the fields of high-dimensional statistics and compressed sensing, we have a clear picture of LASSO's performance. Second, it is convex and efficient algorithms exist for finding its global minima. Unfortunately, neither of the above two properties hold for 0 ≤ p <; 1. However, they are still appealing because of the following folklores in the high-dimensional statistics. First, x̂(γ, p) is closer to xo than x̂(γ, 1). Second, if we employ iterative methods that aim to converge to a local minima of arg minx (1/2)∥y - Ax∥22 + γ∥x∥pp, then under good initialization, these algorithms converge to a solution that is still closer to xo than x̂(γ, 1). In spite of the existence of plenty of empirical results that support these folklore theorems, the theoretical progress to establish them has been very limited. This paper aims to study the above-mentioned folklore theorems and establish their scope of validity. Starting with approximate message passing (AMP) algorithm as a heuristic method for solving ℓp-regularized least squares, we study the following questions. First, what is the impact of initialization on the performance of the algorithm? Second, when does the algorithm recover the sparse signal xo under a “good” initialization? Third, when does the algorithm converge to the sparse signal regardless of the initia...
Published in: IEEE Transactions on Information Theory ( Volume: 63, Issue: 11, November 2017)