Skip to main content
SpringerLink
Log in

A fast splitting method tailored for Dantzig selector

  • Original Paper
  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

In this paper, we introduce a splitting method for solving Dantzig selector problem, a new linear regression model that was extensively studied in the literature in the past few years. The new method is very simple in the sense that, per iteration, it only performs a projection onto a box, and does some matrix-vector products. We prove the global convergence of the method and report some promising numerical results, which demonstrate that the new method is competitive with some state-of-the-art methods recently developed in the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Becker, S., Candés, E., Grant, M.: Templates for convex cone problems with applications to sparse signal recovery. Math. Program. Comput. 3, 165–218 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  2. Bertsekas, D., Tsitsiklis, J.: Parallel and Distributed Computation, Numerical Methods. Prentice-Hall, Englewood Cliffs (1989)

    MATH  Google Scholar 

  3. Bickel, P.: Discussion: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann. Stat. 35(6), 2352–2357 (2007)

    Article  MathSciNet  Google Scholar 

  4. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2010)

    Article  MATH  Google Scholar 

  5. Breiman, L.: Better subset regression using the non-negative garrote. Technometrics 37, 373–384 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  6. Cai, T., Lv, J.: Discussion: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann. Stat. 35, 2365–2368 (2007)

    Article  MathSciNet  Google Scholar 

  7. Candés, E., Tao, T.: The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann. Stat. 35, 2313–2351 (2007)

    Article  MATH  Google Scholar 

  8. Candés, E., Tao, T.: Rejoinder: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann. Stat. 35, 2392–2404 (2007)

    Article  Google Scholar 

  9. Chan, R., Yang, J., Yuan, X.: Alternating direction method for image inpainting in wavelet domain. SIAM J. Imaging Sci. 4, 807–826 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  10. Chen, C., He, B., Yuan, X.: Matrix completion via alternating direction method. IMA J. Numer. Anal. 32, 227–245 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  11. Dobra, A.: Variable selection and dependency networks for genomewide data. Biostatistics 10, 621–639 (2009)

    Article  Google Scholar 

  12. Donoho, D., Tsaig, Y.: Fast solution of \(\ell _1\) minimization problems when the solution may be sparse. IEEE Trans. Inf. Theory 54, 4789–4812 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  13. Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32, 407–451 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  14. Efron, B., Hastie, T., Tibshirani, R.: Discussion: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann. Stat. 35, 2358–2364 (2007)

    Article  MathSciNet  Google Scholar 

  15. Esser, E., Zhang, X., Chan, T.: A general framework for a class of first order primal-dual aalgorithm for TV minimization. SIAM J. Imaging Sci. 3, 1015–1046 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  16. Facchinei, F., Pang, J.: Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer, New York (2003)

    Google Scholar 

  17. Friedlander, M., Saunders, M.: Discussion: the dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann. Stat. 35, 2385–2391 (2007)

    Article  MathSciNet  Google Scholar 

  18. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Comput. Math. Appl. 2, 16–40 (1976)

    Article  Google Scholar 

  19. Golub, T., Slonim, D., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J., Coller, H., Loh, M., Downing, J., Caligiuri, M., Bloomfield, C., Lander, E.: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286, 531–537 (1999)

    Article  Google Scholar 

  20. Han, D., He, H., Yang, H., Yuan, X.: A customized Douglas-Rachford splitting algorithm for separable convex minimization with linear constraints. Numer. Math. 127, 167–200 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  21. He, B., Xu, M., Yuan, X.: Solving large-scale least squares covariance matrix problems by alternating direction methods. SIAM J. Matrix Anal. Appl. 32, 136–152 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  22. Hestenes, M.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)

    Article  MATH  MathSciNet  Google Scholar 

  23. James, G., Radchenko, P., Lv, J.: DASSO: connections between the Dantzig selector and lasso. J. R. Stat. Soc. B 71, 127–142 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  24. Lu, Z.: Primal-dual first-order methods for a class of cone programming. Optim. Method Softw. 28, 1262–1281 (2013)

    Article  MATH  Google Scholar 

  25. Lu, Z., Pong, T., Zhang, Y.: An alternating direction method for finding Dantzig selectors. Comput. Stat. Data Anal. 56, 4037–4046 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  26. Meinshausen, N., Rocha, G., Yu, B.: Discussion: a tale of three cousins: Lasso, L2Boosting and Dantzig. Ann. Stat. 35(6), 2373–2384 (2007)

    Article  MathSciNet  Google Scholar 

  27. Powell, M.: A method for nonlinear constraints in minimization problems. In: R. Fletcher (ed.) Optimization, pp. 283–298. Academic Press, New York (1969)

  28. Ritov, Y.: Discussion: the Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann. Stat. 35, 2370–2372 (2007)

    Article  MathSciNet  Google Scholar 

  29. Romberg, J.: The Dantzig selector and generalized thresholding. In: IEEE 42nd Annual Conference on Information Sciences and Systems, pp. 22–25. Princeton, New Jersey (2008)

  30. Tibshirani, R.: Regression shrinkage and selection via the LASSO. J. R. Stat. Soc. B 58, 267–288 (1996)

    MATH  MathSciNet  Google Scholar 

  31. Wang, X., Yuan, X.: The linearized alternating direction method of multipliers for Dantzig selector. SIAM J. Sci. Comput. 34, A2792–A2811 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  32. Yang, J., Zhang, Y.: Alternating direction algorithms for \(\ell _1\)-problems in compressive sensing. SIAM J. Sci. Comput. 332, 250–278 (2011)

    Article  Google Scholar 

  33. Yeung, K., Bumgarner, R., Raftery, A.: Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray data. Bioinformatics 21, 2394–2402 (2005)

    Article  Google Scholar 

  34. Yuan, X.: Alternating direction methods for covariance selection models. J. Sci. Comput. 51, 261–273 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  35. Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. B 67, 301–320 (2005)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

We are grateful to the anonymous referees for their close reading, valuable and constructive comments on earlier versions of this paper. We also would like to thank the authors of [25, 31] for sharing their code to us, and special thanks go to Prof. Zhaosong Lu and Prof. Tingkei Pong for their help on the numerical experiments for real data. The research of the first author was supported by NSFC (11301123) and Zhejiang Provincial NSFC (LZ14A010003); The research of the second author was supported by NSFC (11401315) and Jiangsu Provincial NSFC (BK20140914); The research of the third author was supported by NSFC (11371197; 11431002). The second and the third authors were also supported by a project funded by PAPD of Jiangsu Higher Education Institutions.

Author information

Authors and Affiliations

  1. Department of Mathematics, School of Science, Hangzhou Dianzi University, Hangzhou, 310018, China

    Hongjin He

  2. School of Mathematical Sciences, Jiangsu Key Laboratory for NSLSCS, Nanjing Normal University, Nanjing, 210023, China

    Xingju Cai & Deren Han

Authors
  1. Hongjin He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xingju Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Deren Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xingju Cai.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, H., Cai, X. & Han, D. A fast splitting method tailored for Dantzig selector. Comput Optim Appl 62, 347–372 (2015). https://doi.org/10.1007/s10589-015-9748-2

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-015-9748-2

Keywords

Search

Navigation