Abstract
There have been several studies to relax a bias problem in LASSO (Least Absolute Shrinkage and Selection Operator). In this article, we considered to solve a bias problem of LASSO estimator by scaling and derived a model selection criterion under the scaling method. The proposed scaling value is valid to compensate the excessive shrinkage of LASSO estimator and is easy to compute by using LASSO estimator. Moreover, we derived SURE (Stein’s Unbiased Risk Estimate) as a model selection criterion. This analytic solution is also a benefit of the proposed scaling value. Furthermore, we verified the risk estimate and confirmed its effectiveness through a simple numerical example.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bühlmann, P., Meier, L.: Discussion: one-step sparse estimates in nonconcave penalized likelihood models. Ann. Stat. 36, 1534–1541 (2008)
Carter, C.K., Eagleson, G.K.: A comparison of variance estimators in nonparametric regression. J. R. Stat. Soc. B 54, 773–780 (1992)
Donoho, D.L., Johnstone, I.M.: Ideal spatial adaptation via wavelet shrinkage. Biometrika 81, 425–455 (1994)
Dossal, C., Kachour, M., Fadili, J., Peyré, G., Chesneau, C.: The degrees of freedom of the LASSO for general design matrix. Statistica Sinica 23, 809–828 (2013)
Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32, 407–499 (2004)
Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Statist. Assoc. 96, 1348–1360 (2001)
Hagiwara, K.: A problem in model selection of LASSO and introduction of scaling. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds.) ICONIP 2016. LNCS, vol. 9948, pp. 20–27. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46672-9_3
Knight, K., Fu, W.: Asymptotics for LASSO-type estimators. Ann. Stat. 28, 1356–1378 (2000)
Leng, C.L., Lin, Y., Wahba, G.: A note on the LASSO and related procedures in model selection. Statistica Sinica 16, 1273–1284 (2006)
Meinshausen, N.: Relaxed LASSO. Comput. Stat. Data Anal. 52, 374–393 (2007)
Stein, C.: Estimation of the mean of a multivariate normal distribution. Ann. Stat. 9, 1135–1151 (1981)
Tibshirani, R.: Regression shrinkage and selection via the LASSO. J. R. Stat. Soc. Ser. B. 58, 267–288 (1996)
Tibshirani, R., Taylor, J.: Degrees of freedom in LASSO problems. Ann. Stat. 40, 1198–1232 (2012)
Wu, T.T., Lange, K.: Coordinate descent algorithms for LASSO penalized regression. Ann. Appl. Stat. 2, 224–244 (2008)
Xiao, N., Xu, Q.S.: Multi-step adaptive elastic-net: reducing false positives in high-dimensional variable selection. J. Stat. Comput. Simul. 85, 3755–3765 (2015)
Zhang, C.H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38, 894–942 (2010)
Zou, H.: The adaptive LASSO and its oracle properties. J. Am. Stat. Assoc. 101, 1418–1492 (2006)
Zou, H., Hastie, T., Tibshirani, R.: On the degree of freedom of the LASSO. Ann. Stat. 35, 2173–2192 (2007)
Acknowledgements
This work was supported in part by Japan Society for the Promotion of Science (JSPS) KAKENHI Grant Number 18K11433.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Hagiwara, K. (2019). A Model Selection Criterion for LASSO Estimate with Scaling. In: Gedeon, T., Wong, K., Lee, M. (eds) Neural Information Processing. ICONIP 2019. Lecture Notes in Computer Science(), vol 11954. Springer, Cham. https://doi.org/10.1007/978-3-030-36711-4_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-36711-4_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36710-7
Online ISBN: 978-3-030-36711-4
eBook Packages: Computer ScienceComputer Science (R0)