Abstract
As two popularly used variable selection methods, the Dantzig selector and the LASSO have been proved asymptotically equivalent in some scenarios. However, it is not the case in general for linear models, as disclosed in Gai, Zhu and Lin’s paper in 2013. In this paper, it is further shown that generally the asymptotic equivalence is not true either for a general single-index model with random design of predictors. To achieve this goal, the authors systematically investigate necessary and sufficient conditions for the consistent model selection of the Dantzig selector. An adaptive Dantzig selector is also recommended for the cases where those conditions are not satisfied. Also, different from existing methods for linear models, no distributional assumption on error term is needed with a trade-off that more stringent condition on the predictor vector is assumed. A small scale simulation is conducted to examine the performances of the Dantzig selector and the adaptive Dantzig selector.
Similar content being viewed by others
References
Härdle W, Hall P, and Ichimura H, Optimal smoothing in single-index models, Ann. Statist., 1993, 21: 157–178.
Xia Y, Tong H, Li W K, and Zhu L, An adaptive estimation of dimension reduction space (with discussions), Journal of the Royal Statistical Society, Series B, 2002, 64: 363–410.
Kong E and Xia Y, Variable selection for the single-index model, Biometrika, 2006, 94: 217–229.
Wang T and Zhu L X, A distribution-based LASSO for a general single-index model, Science China Mathematics, 2015, 58: 109–130.
Tibshirani R, Regression shrinkage and selection via the Lasso, Journal of the Royal Statistical Society, Series B, 1996, 58: 267–288.
Candès E and Tao T, The Dantzig selector: Statistical estimation when p is much larger than n, Annals of Statistics, 2007, 35: 2313–2351.
Dicker L and Lin X H, Parallelism, uniqueness, and large-sample asymptotics for the Dantzig selector, Canadian Journal of Statistics, 2013, 41: 23–35.
Gai Y J, Zhu L X, and Lin L, Model selection consistency of Dantzig selector, Statistica Sinica, 2013, 23: 615–634.
Bickel P J, Ritov Y, and Tsybakov A B, Simultaneous analysis of Lasso and Dantzig selector, Annals of Statistics, 2009, 37: 1705–1732.
Asif M S and Romberg J, On the lasso and dantzig selector equivalence, 44th Annual Conference on Information Sciences and Systems (CISS), 2010.
Zhao P and Yu B, On model selection consistency of LASSO, The Journal of Machine Learning Research, 2006, 7: 2541–2563.
Zou H, The adaptive LASSO and its oracle properties, J. Amer. Statist. Asoc., 2006, 101: 1418–1429.
Li K C, Sliced inverse regression for dimension reduction (with discussion), J. Amer. Statist. Assoc., 1991, 86: 316–327.
Fan J and Lü J, Sure independence screening for ultrahigh dimensional feature space, Journal of the Royal Statistical Society, Series B, 2008, 70: 849–911.
Lin Z Y, Xiang Y B, and Zhang C Y, Adaptive lasso in high-dimensional settings, Journal of Nonparametric Statistics, 2009, 21: 683–696.
Huang J, Horowitz J L, and Ma S, Asymptotic properties of bridge estimators in sparse highdimensional regression models, Annals of Statistics, 2008, 36: 587–613.
Golub T R, Slonim D K, Tamayo P, Hurad C, Gaasenbeek M, Mesirov J P, Coller H, Loh M L, Downing J R, Caligiuri M A, Bloomfield C D, and Lander E S, Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring, Science, 1999, 286: 531–537.
Dettling M, Bagboosting for tumor classification with gene expression data, Bioinformatics, 2004, 20: 3583–3593.
Dudoit S, Fridlyand J, and Speed T P, Comparison of discrimination methods for the classification of tumors using gene expression data, J. Amer. Statist. Assoc., 2002, 97: 77–87.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was supported by the National Natural Science Foundation of China under Grant Nos. 11501354, 11201499, 11301309 and 71473280, and 2015 Shanghai Young Faculty Training Program under Grant No. A1A-6119-15-003.
This paper was recommended for publication by Editor SUN Liuquan.
Rights and permissions
About this article
Cite this article
Fan, Y., Gai, Y. & Zhu, L. Asymtotics of Dantzig selector for a general single-index model. J Syst Sci Complex 29, 1123–1144 (2016). https://doi.org/10.1007/s11424-015-4124-y
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11424-015-4124-y