Skip to main content
SpringerLink
Log in

On discrete ℓ1-regularization

  • Published:
Advances in Computational Mathematics Aims and scope Submit manuscript

Abstract

A real m × n matrix A and a vector y ∈ ℝm determine the discrete l 1-regularization (DLR) problem

$$ \min \left\{\mbox{\,}|y-Ax|_1+\rho |x|_1:\,x\in\mathbb{R}^n \right\}, $$
(0.1)

where | · |1 denotes the l 1-norm of a vector and ρ ≥ 0 is a nonnegative parameter. In this paper, we provide a detailed analysis of this problem which include a characterization of all solutions to (0.1), remarks about the geometry of the solution set and an effective iterative algorithm for numerical solution of (0.1). We are specially interested in the behavior of the solution of (0.1) as a function of ρ and in this regard, we prove in general the existence of critical values of ρ between which the l 1-norm of any solution remains constant. These general remarks are significantly refined when A is a strictly totally positive (STP) matrix. The importance of STP matrices is well-established [5, 14]. Under this setting, the relationship between the number of nonzero coordinates of a distinguished solution of the DLR problem is precisely explained as a function of the regularization parameter for a certain class of vectors in ℝm. Throughout our analysis of the DLR problem, we emphasize the importance of the dual maximum problem by demonstrating that any solution of it leads to a solution of the DLR problem, and vice versa.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abdelmalek, N.N.: Linear L 1 approximation for a discrete point set and L 1 solutions of overdetermined linear equations. J. Assoc. Comput. Mach. 18, 41–47 (1971)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Belmont, MA (2003)

    Google Scholar 

  3. Cheney, E.W.: Introduction to Approximation Theory. AMS Chelsea Publishing, Providence, RI (1998)

    MATH  Google Scholar 

  4. Donoho, D.L.: For most large underdetermined systems of linear equations the minimal l 1-norm solution is also the sparsest solution. Commun. Pure Appl. Math. 59, 797–829 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  5. Fallat, S.M., Johnson, C.R.: Totally Nonnegative Matrices. Princeton University Press, Princeton, NJ (2011)

    MATH  Google Scholar 

  6. Florenzano, M., Le Van, C.: Finite Dimensional Convexity and Optimization. Springer, New York (2001)

    Book  MATH  Google Scholar 

  7. Fu, H., Ng, M.K., Nikolova, M., Barlow, J.L.: Efficient minimization methods of mixed l2-l1 and l1-l1 norms for image restoration. SIAM J. Sci. Comput. 27, 1881–1902 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  8. Gasca, M., Micchelli, C.A.: Total Positivity and its Applications. Kluwer, Norwell, MA (1996)

    MATH  Google Scholar 

  9. Karlin, S.: Total Positivity, vol. I. Stanford University Press, Stanford, CA (1968)

    Google Scholar 

  10. Kress, R.: Numerical Analysis. Springer, New York (1998)

    Book  MATH  Google Scholar 

  11. Micchelli, C.A., Pinkus, A.: On n-widths in L  ∞ . Trans. Am. Math. Soc. 234, 139–174 (1977)

    MathSciNet  MATH  Google Scholar 

  12. Micchelli, C.A., Pinkus, A.: On n-widths in l  ∞  II. Some related extremal problems. Constr. Func. Theory 77, 403–416 (1980)

    Google Scholar 

  13. Micchelli, C.A., Shen, L., Xu, Y., Zeng, X.: Proximity Algorithms for Image Models II: L1/ TV Denoising (preprint)

  14. Pinkus, A.: Totally Positive Matrices. Cambridge University Press, Cambridge, UK (2010)

    MATH  Google Scholar 

  15. Pinkus, A.: Sparse representations and approximation theory. J. Approx. Theory 163, 388–412 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  16. Pinkus, A.: On L 1-Approximation. Cambridge University Press, Cambridge, UK (1989)

    Google Scholar 

  17. Schmidt, D.: Almost Chebyshev properties for L 1-approximation of continuous functions. J. Approx. Theory 61, 335–350 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  18. Yang, J., Zhang, Y.: Alternating direction algorithms for ℓ1-problems in compressive sensing. SIAM J. Sci. Comput. 33, 250–278 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  19. Zhang, Y.: On theory of compressive sensing via l 1 minimization: simple derivations and extensions. Technical Report TR08-11. Dept. of Computational and Applied Mathematics, Rice Univ. (2008)

  20. Zhao, T.: l 1-regularization, duality and strictly total positivity. PhD thesis, School of Mathematics and Computational Science, Sun Yat-sen Univ. (in preparation)

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, City University of Hong Kong, Tat Chee Avenue, Kowloon, Hong Kong, China

    Charles A. Micchelli

  2. Department of Mathematics and Statistics, State University of New York, The University at Albany, Albany, NY, 12222, USA

    Charles A. Micchelli

  3. Guangdong Province Key Lab of Computational Science and School of Mathematics and Computational Science, Sun Yat-sen University, Guangzhou, China

    Tao Zhao

Authors
  1. Charles A. Micchelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Zhao.

Additional information

Communicated by Yuesheng Xu.

C. A. Micchelli was partially supported by the US Air Force Office of Scientific Research under grant FA9550-09-1-0511 and by the US National Science Foundation under grants DMS-0712827 and DMS-1115523.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Micchelli, C.A., Zhao, T. On discrete ℓ1-regularization. Adv Comput Math 38, 441–454 (2013). https://doi.org/10.1007/s10444-011-9247-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10444-011-9247-7

Keywords

Mathematics Subject Classifications (2010)

Search

Navigation