Abstract
When there are outliers or heavy-tailed distributions in the data, the traditional least squares with penalty function is no longer applicable. In addition, with the rapid development of science and technology, a lot of data, enjoying high dimension, strong correlation and redundancy, has been generated in real life. So it is necessary to find an effective variable selection method for dealing with collinearity based on the robust method. This paper proposes a penalized M-estimation method based on standard error adjusted adaptive elastic-net, which uses M-estimators and the corresponding standard errors as weights. The consistency and asymptotic normality of this method are proved theoretically. For the regularization in high-dimensional space, the authors use the multi-step adaptive elastic-net to reduce the dimension to a relatively large scale which is less than the sample size, and then use the proposed method to select variables and estimate parameters. Finally, the authors carry out simulation studies and two real data analysis to examine the finite sample performance of the proposed method. The results show that the proposed method has some advantages over other commonly used methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Frank I E and Friedman J H, A statistical view of some chemometrics regression tools (with discussion), Technometrics, 1993, 35: 109–135.
Tibshirani R J, Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society, Series B, 1996, 73: 273–282.
Efron B, Hastie T, and Tibshirani J R, Least angle regression, The Annals of Statistics, 2004, 32: 407–451.
Zou H, The adaptive lasso and its oracle properties, Journal of the American Statistical Association, 2006, 101: 1418–1429.
Qian W and Yang Y, Model selection via standard error adjusted adaptive lasso, Annals of the Institute of Statistical Mathematics, 2013, 65: 295–318.
Zou H and Hastie T, Regularization and variable selection via the elastic net, Journal of the Royal Statistical Society, Series B, 2005, 67: 301–320.
Zou H and Zhang H H, On the adaptive elastic-net with a diverging number of parameters, The Annals of Statistics, 2009, 37: 1733–1751.
Huber P J, Robust estimation of a location parameter, The Annals of Statistics, 1964, 35: 73–101.
Yohai V J and Maronna R A, Asymptotic behavior of M-estimators for the linearmodel, The Annals of Statistics, 1979, 7: 258–268.
Huber P J, Robust regression: Asymptotics, conjectures and Monte Carlo, The Annals of Statistics, 1973, 1: 799–821.
Portnoy S, Asymptotic behavior of M-estimatiors of p regression parameters when p2/n is large I Consistency, The Annals of Statistics, 1984, 12: 1298–1309.
Portnoy S, Asymptotic behavior of M-estimators of p regression parameters when p2/n is large II Normal approximation, The Annals of Statistics, 1985, 13: 1403–1417.
Welsh A H, On M-processes and M-estimation, The Annals of Statistics, 1989, 17: 337–361.
Wu Y and Zen M M, A strong consistent information criterion for linear model selection based on M-estimation, Probability Theory and Related Fields, 1999, 113: 599–625.
Zheng G, Freidlin B, and Gastwirth J L, Using Kullback-Leibler information for model selection when the data-generating model is unknown: Applications to genetic testing problems, Statistica Sinica, 2004, 14: 1021–1036.
Li G, Peng H, and Zhu L, Nonconcave penalized M-estimation with a diverging number of parameters, Statistica Sinica, 2011, 21: 391–419.
Zhao W and Zhang R, Variable selection of varying dispersion student-t regression models. Journal of Systems Science and Complexity, 2015, 28(4): 961–977.
Xiao N and Xu Q S, Multi-step adaptive elastic-net: Reducing false postives in high-dimensional variable selection, Journal of Statistical Computation and Simulation, 2015, 85: 1–11.
Bai Z D, Rao C R, and Wu Y, M-estimation of multivariate linear regression parameters under a convex discrepancy function, Statistica Sinica, 1992, 2: 237–254.
Wu W B, M-estimation of linear models with dependent errors, The Annals of Statistics, 2007, 35: 495–521.
Schwarz G, Estimating the dimension of a model, The Annals of Statistics, 1978, 6: 461–464.
Candes E and Tao T, The Dantzig selector: Statistical estimation when p is much larger than n. (with discussion), The Annals of Statistics, 2005, 35: 2313–2351.
Fan J and Lü J, Sure independence screening for ultra-high dimensional feature space (with discussion), Journal of the Royal Statistical Society, Series B, 2008, 70: 849–911.
Harrison D and Rubinfeld D L, Hedonic housing prices and the demand for clean air, Journal of Environmental Economics and Management, 1978, 5: 81–102.
Li X, Zhao T, Yuan X, et al., The flare package for high dimensional linear regression and precision matrix estimation in R, Journal of Machine Learning Research, 2015, 16: 553–557.
Scheetz T E, Kim K Y, Swiderski R E, et al., Regulation of gene expression in the mammalian eye and its relevance to eye disease, Proceedings of the National Academy of Sciences of the United States of America, 2006, 103: 14429–14434.
Bai Z D, Rao C R, and Wu Y, Limiting behavior of M-estimators of regression coefficients in high dimensional linear models I. scale dependent case, Journal of Multivariate Analysis, 1994, 51: 211–239.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was supported by the National Natural Science Foundation of China under Grant Nos. 12271294, 12171225 and 12071248.
Rights and permissions
About this article
Cite this article
Wu, X., Wang, M., Hu, W. et al. Penalized M-Estimation Based on Standard Error Adjusted Adaptive Elastic-Net. J Syst Sci Complex 36, 1265–1284 (2023). https://doi.org/10.1007/s11424-023-1400-0
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11424-023-1400-0