Skip to main content
Log in

A partially proximal linearized alternating minimization method for finding Dantzig selectors

  • Published:
Computational and Applied Mathematics Aims and scope Submit manuscript

Abstract

The Dantzig selector (DS) is an efficient estimator designed for high-dimensional linear regression problems, especially for the case where the number of samples n is much less than the dimension of features (or variables) p. In this paper, we first reformulate the underlying DS model as an unconstrained minimization problem of the sum of two nonsmooth convex functions and a smooth coupled function. Then by exploiting the structure of the resulting model, we propose a partially proximal linearized alternating minimization method (P-PLAM), whose two subproblems are easy enough with closed-form solutions. Another remarkable advantage is that P-PLAM requires only one starting point, which is potentially helpful for saving computing time. A series of computational experiments on synthetic and real-world data sets demonstrate that the proposed P-PLAM has promising numerical performance in the sense that P-PLAM can obtain higher quality solutions by taking less computing time than some existing state-of-the-art first-order solvers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Attouch H, Bolte J, Redont P, Soubeyran A (2010) Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka–Lojasiewicz inequality. Math Oper Res 35:438–457

    Article  MathSciNet  Google Scholar 

  • Beck A (2015) On the convergence of alternating minimization for convex programming with applications to iterative reweighted least squares and decomposition schemes. SIAM J Optim 25:185–209

    Article  MathSciNet  Google Scholar 

  • Becker S, Candés E, Grant M (2011) Templates for convex cone problems with applications to sparse signal recovery. Math Program Comput 3:165–218

    Article  MathSciNet  Google Scholar 

  • Bickel P (2007) Discussion: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann Stat 35(6):2352–2357

    Article  Google Scholar 

  • Bolte J, Sabach S, Teboulle M (2014) Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math Program 146:459–494

    Article  MathSciNet  Google Scholar 

  • Breiman L (1995) Better subset regression using the non-negative garrote. Technometrics 37:373–384

    Article  MathSciNet  Google Scholar 

  • Cai T, Lv J (2007) Discussion: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann Stat 35:2365–2368

    Article  Google Scholar 

  • Candés E, Tao T (2007) The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann Stat 35:2313–2351

    Article  MathSciNet  Google Scholar 

  • Candés E, Tao T (2007) Rejoinder: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann Stat 35:2392–2404

    Article  Google Scholar 

  • Dobra A (2009) Variable selection and dependency networks for genomewide data. Biostatistics 10:621–639

    Article  Google Scholar 

  • Efron B, Hastie T, Johnstone I, Tibshirani R (2004) Least angle regression. Ann Stat 32:407–451

    Article  MathSciNet  Google Scholar 

  • Efron B, Hastie T, Tibshirani R (2007) Discussion: The Dantzig selector: Statistical estimation when \(p\) is much larger than \(n\). Ann Stat 35:2358–2364

    Article  Google Scholar 

  • Friedlander M, Saunders M (2007) Discussion: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann Stat 35:2385–2391

    Article  Google Scholar 

  • Golub T, Slonim D, Tamayo P, Huard C, Gaasenbeek M, Mesirov J, Coller H, Loh M, Downing J, Caligiuri M, Bloomfield C, Lander E (1999) Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286:531–537

    Article  Google Scholar 

  • Gu G, He B, Yuan X (2014) Customized proximal point algorithms for linearly constrained convex minimization and saddle-point problems: a unified approach. Comput. Optim. Appl. 59:135–161

    Article  MathSciNet  Google Scholar 

  • He H, Cai X, Han D (2015) A fast splitting method tailored for Dantzig selectors. Comput Optim Appl 62:347–372

    Article  MathSciNet  Google Scholar 

  • He H, Xu HK (2017) Splitting methods for split feasibility problems with application to Dantzig selectors. Inverse Probl. 33:055,003 (28pp)

  • James G, Radchenko P, Lv J (2009) DASSO: connections between the Dantzig selector and lasso. J R Stat Soc B 71:127–142

    Article  MathSciNet  Google Scholar 

  • Jia J, Rohe K (2015) Preconditioning the Lasso for sign consistency. Electronic Journal of Statistics 9:1150–1172

    Article  MathSciNet  Google Scholar 

  • Lu Z, Pong T, Zhang Y (2012) An alternating direction method for finding Dantzig selectors. Comput Stat Data Anal 56:4037–4046

    Article  MathSciNet  Google Scholar 

  • Meinshausen N, Rocha G, Yu B (2007) Discussion: A tale of three cousins: Lasso, L2Boosting and Dantzig. Ann Stat 35(6):2373–2384

    Article  Google Scholar 

  • Nocedal J, Wright S (2006) Numerical Optimization, 2nd edn. Springer Series in Operations Research and Financial Engineering. Springer, New York

  • Prater A, Shen L, Suter B (2015) Finding Dantzig selectors with a proximity operator based fixed-point algorithm. Comput Stat Data Anal 90:36–46

    Article  MathSciNet  Google Scholar 

  • Ritov Y (2007) Discussion: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann Stat 35:2370–2372

    Article  Google Scholar 

  • Shefi R, Teboulle M (2016) On the rate of convergence of the proximal alternating linearized minimization algorithm for convex problems. Eur J Comput Optim 4:27–46

    Article  MathSciNet  Google Scholar 

  • Tibshirani R (1996) Regression shrinkage and selection via the LASSO. J R Stat Soc Ser B 58:267–288

    MathSciNet  MATH  Google Scholar 

  • Wang X, Yuan X (2012) The linearized alternating direction method of multipliers for Dantzig selector. SIAM J Sci Comput 34:A2792–A2811

    Article  MathSciNet  Google Scholar 

  • Yeung K, Bumgarner R, Raftery A (2005) Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray data. Bioinformatics 21:2394–2402

    Article  Google Scholar 

  • Zhao P, Yu B (2006) On model selection consistency of LASSO. J Mach Learn Res 7:2541–2563

    MathSciNet  MATH  Google Scholar 

  • Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J R Stat Soc B 67:301–320

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the editor for his valuable comments, which helped us improve the presentation of this paper. This work was supported in part by National Natural Science Foundation of China (Nos. 11771113 and U1811461) and Zhejiang Provincial Natural Science Foundation of China at Grant No. LY20A010018.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hongjin He.

Additional information

Communicated by Gabriel Haeser.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mao, X., He, H. & Xu, HK. A partially proximal linearized alternating minimization method for finding Dantzig selectors. Comp. Appl. Math. 40, 62 (2021). https://doi.org/10.1007/s40314-021-01450-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s40314-021-01450-5

Keywords

Mathematics Subject Classification

Navigation