Abstract
The Dantzig selector (DS) is an efficient estimator designed for high-dimensional linear regression problems, especially for the case where the number of samples n is much less than the dimension of features (or variables) p. In this paper, we first reformulate the underlying DS model as an unconstrained minimization problem of the sum of two nonsmooth convex functions and a smooth coupled function. Then by exploiting the structure of the resulting model, we propose a partially proximal linearized alternating minimization method (P-PLAM), whose two subproblems are easy enough with closed-form solutions. Another remarkable advantage is that P-PLAM requires only one starting point, which is potentially helpful for saving computing time. A series of computational experiments on synthetic and real-world data sets demonstrate that the proposed P-PLAM has promising numerical performance in the sense that P-PLAM can obtain higher quality solutions by taking less computing time than some existing state-of-the-art first-order solvers.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs40314-021-01450-5/MediaObjects/40314_2021_1450_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs40314-021-01450-5/MediaObjects/40314_2021_1450_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs40314-021-01450-5/MediaObjects/40314_2021_1450_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs40314-021-01450-5/MediaObjects/40314_2021_1450_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs40314-021-01450-5/MediaObjects/40314_2021_1450_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs40314-021-01450-5/MediaObjects/40314_2021_1450_Fig6_HTML.png)
Similar content being viewed by others
References
Attouch H, Bolte J, Redont P, Soubeyran A (2010) Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka–Lojasiewicz inequality. Math Oper Res 35:438–457
Beck A (2015) On the convergence of alternating minimization for convex programming with applications to iterative reweighted least squares and decomposition schemes. SIAM J Optim 25:185–209
Becker S, Candés E, Grant M (2011) Templates for convex cone problems with applications to sparse signal recovery. Math Program Comput 3:165–218
Bickel P (2007) Discussion: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann Stat 35(6):2352–2357
Bolte J, Sabach S, Teboulle M (2014) Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math Program 146:459–494
Breiman L (1995) Better subset regression using the non-negative garrote. Technometrics 37:373–384
Cai T, Lv J (2007) Discussion: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann Stat 35:2365–2368
Candés E, Tao T (2007) The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann Stat 35:2313–2351
Candés E, Tao T (2007) Rejoinder: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann Stat 35:2392–2404
Dobra A (2009) Variable selection and dependency networks for genomewide data. Biostatistics 10:621–639
Efron B, Hastie T, Johnstone I, Tibshirani R (2004) Least angle regression. Ann Stat 32:407–451
Efron B, Hastie T, Tibshirani R (2007) Discussion: The Dantzig selector: Statistical estimation when \(p\) is much larger than \(n\). Ann Stat 35:2358–2364
Friedlander M, Saunders M (2007) Discussion: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann Stat 35:2385–2391
Golub T, Slonim D, Tamayo P, Huard C, Gaasenbeek M, Mesirov J, Coller H, Loh M, Downing J, Caligiuri M, Bloomfield C, Lander E (1999) Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286:531–537
Gu G, He B, Yuan X (2014) Customized proximal point algorithms for linearly constrained convex minimization and saddle-point problems: a unified approach. Comput. Optim. Appl. 59:135–161
He H, Cai X, Han D (2015) A fast splitting method tailored for Dantzig selectors. Comput Optim Appl 62:347–372
He H, Xu HK (2017) Splitting methods for split feasibility problems with application to Dantzig selectors. Inverse Probl. 33:055,003 (28pp)
James G, Radchenko P, Lv J (2009) DASSO: connections between the Dantzig selector and lasso. J R Stat Soc B 71:127–142
Jia J, Rohe K (2015) Preconditioning the Lasso for sign consistency. Electronic Journal of Statistics 9:1150–1172
Lu Z, Pong T, Zhang Y (2012) An alternating direction method for finding Dantzig selectors. Comput Stat Data Anal 56:4037–4046
Meinshausen N, Rocha G, Yu B (2007) Discussion: A tale of three cousins: Lasso, L2Boosting and Dantzig. Ann Stat 35(6):2373–2384
Nocedal J, Wright S (2006) Numerical Optimization, 2nd edn. Springer Series in Operations Research and Financial Engineering. Springer, New York
Prater A, Shen L, Suter B (2015) Finding Dantzig selectors with a proximity operator based fixed-point algorithm. Comput Stat Data Anal 90:36–46
Ritov Y (2007) Discussion: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann Stat 35:2370–2372
Shefi R, Teboulle M (2016) On the rate of convergence of the proximal alternating linearized minimization algorithm for convex problems. Eur J Comput Optim 4:27–46
Tibshirani R (1996) Regression shrinkage and selection via the LASSO. J R Stat Soc Ser B 58:267–288
Wang X, Yuan X (2012) The linearized alternating direction method of multipliers for Dantzig selector. SIAM J Sci Comput 34:A2792–A2811
Yeung K, Bumgarner R, Raftery A (2005) Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray data. Bioinformatics 21:2394–2402
Zhao P, Yu B (2006) On model selection consistency of LASSO. J Mach Learn Res 7:2541–2563
Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J R Stat Soc B 67:301–320
Acknowledgements
The authors would like to thank the editor for his valuable comments, which helped us improve the presentation of this paper. This work was supported in part by National Natural Science Foundation of China (Nos. 11771113 and U1811461) and Zhejiang Provincial Natural Science Foundation of China at Grant No. LY20A010018.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Gabriel Haeser.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Mao, X., He, H. & Xu, HK. A partially proximal linearized alternating minimization method for finding Dantzig selectors. Comp. Appl. Math. 40, 62 (2021). https://doi.org/10.1007/s40314-021-01450-5
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s40314-021-01450-5