Loading [a11y]/accessibility-menu.js
Learning Mixtures of Sparse Linear Regressions Using Sparse Graph Codes | IEEE Journals & Magazine | IEEE Xplore

Learning Mixtures of Sparse Linear Regressions Using Sparse Graph Codes


Abstract:

In this paper, we consider the mixture of sparse linear regressions model. Let β(1), . . ., β(L) ∈ ℂn be L unknown sparse parameter vectors with a total of K non-zero ele...Show More

Abstract:

In this paper, we consider the mixture of sparse linear regressions model. Let β(1), . . ., β(L) ∈ ℂn be L unknown sparse parameter vectors with a total of K non-zero elements. Noisy linear measurements are obtained in the form yi = xiH β(ℓi) + wi, each of which is generated randomly from one of the sparse vectors with the label ℓi unknown. The goal is to estimate the parameter vectors efficiently with low sample and computational costs. This problem presents significant challenges as one needs to simultaneously solve the demixing problem of recovering the labels ℓi as well as the estimation problem of recovering the sparse vectors β(ℓ). Our solution to the problem leverages the connection between modern coding theory and statistical inference. We introduce a new algorithm, MixedColoring, which samples the mixture strategically using query vectors xi constructed based on ideas from sparse graph codes. Our novel code design allows for both efficient demixing and parameter estimation. To find K non-zero elements, it is clear that we need at least Θ(K) measurements, and thus the time complexity is at least (K). In the noiseless setting, for a constant number of sparse parameter vectors, our algorithm achieves the order-optimal sample and time complexities of Θ(K). In the presence of Gaussian noise,1 for the problem with two parameter vectors (i.e., L = 2), we show that the Robust Mixed-Coloring algorithm achieves near-optimal Θ(K polylog(n)) sample and time complexities. When K = O(nα) for some constant α ∈ (0, 1) (i.e., K is sublinear in n), we can achieve sample and time complexities both sublinear in the ambient dimension. In one of our experiments, to recover a mixture of two regressions with dimension n = 500 and sparsity K = 50, our algorithm is more than 300 times faster than EM algorithm, with about one third of its sample cost.
Published in: IEEE Transactions on Information Theory ( Volume: 65, Issue: 3, March 2019)
Page(s): 1430 - 1451
Date of Publication: 08 August 2018

ISSN Information:

Funding Agency:


References

References is not available for this document.