skip to main content
10.1145/3460569.3460578acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicmaiConference Proceedingsconference-collections
research-article

Sparse Logistic Regression with the Hybrid L1/2+1 Regularization

Published: 31 August 2021 Publication History

Abstract

In this paper, L1/2+1 regularized logistic regression model and corresponding algorithm are proposed. The L1/2 regular term has unbiased, sparsity and Oracle properties. The L1 regular term guarantees the convex function characteristics in theory. The regular term of the model is composed of the linear combination of L1/2 norm and L1 norm, which can effectively improve the over fitting problem and the generalization ability of the model. In this algorithm, the idea of coordinate descent method is adopted, and the solution of parameters is transformed into a series of extremum problems of one variable function, thus the analytical expression of parameter estimation is given. Experiments on simulated data and real data show that, in some cases, the model and algorithm proposed in this paper are better than the traditional logistic regression and several classical regularized logistic regression in variable selection and prediction ability, and are suitable for small sample data sets with low correlation between variables.

References

[1]
Thompson E, Bowling B, and Markle R. 2018. Predicting Student Success in a Major's Introductory Biology Course via Logistic Regression Analysis of Scientific Reasoning Ability and Mathematics Scores. J. Research in Science Education. 48 (February 2018): 151-163. https://doi.org/10.1007/s11165-016-9563-5
[2]
David MC, Van Der Pols JC, Williams GM, 2013. Risk of attrition in a longitudinal study of skin cancer: logistic and survival models can give different results. J. Journal of clinical epidemiology. 66 (August 2013): 888-895. https://doi.org/10.1016/j.jclinepi.2013.03.008
[3]
Gajović V, Kerkez M, and Kočović J. 2018. Modeling and simulation of logistic processes: risk assessment with a fuzzy logic technique. J. Simulation (San Diego, Calif.). 94 (June 2018): 507-518. https://doi.org/10.1177/0037549717738351
[4]
Hoerl A, Kennard R. 1970. Ridge Regression: Biased Estimation for Nonorthogonal Problems. J. Technometrics. 12 (1): 55-67. https://doi.org/10.1080/00401706.1970.10488634
[5]
Tibshirani R. 1996. Regression Shrinkage and Selection Via the Lasso. J. Journal of the Royal Statistical Society: Series B. 58 (January 1996): 229-243. https://doi.org/10.1111/j.2517-6161.1996.tb02080.x
[6]
Fan J, Li R. 2011. Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties. J. Journal of the American Statistical Association. 96 (December 2011): 1348-1360. https://doi.org/10.1198/016214501753382273
[7]
Zou H, Hastie T. 2005. Regularization and Variable Selection via the Elastic Net. J. Journal of the Royal Statistical Society: Series B (Statistical Methodology). 67 (April 2005): 301-320. https://doi.org/10.1111/j.1467-9868.2005.00503.x
[8]
Kim SH. 2003. An Investigation of Bayes Estimation Procedures for the Two-Parameter Logistic Model. J. Springer Japan. 389-396. https://doi.org/10.1007/978-4-431-66996-8_44
[9]
Friedman J, Hastie T, and Tibshirani R. 2010. Regularization Paths for Generalized Linear Models via Coordinate Descent. J. Journal of Statistical Software. 33 (February 2010), 1-22. https://doi.org/10.18637/jss.v033.i01
[10]
Yuan Guoxun, Ho C, and Lin C. 2011. An Improved GLMNET for L1-Regularized Logistic Regression. J. ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. San Diego, USA. (August 2011), 33-41. https://doi.org/10.1145/2020408.2020421
[11]
Balamurugan P. 2013. Large-Scale Elastic Net Regularized Linear Classification SVMs and Logistic Regression. J. ACM Sigkdd International Conference on Knowledge Discovery & Data Mining. Dallas, USA. (December 2013). 10.1109/ICDM.2013.126
[12]
Bach F. 2013. Adaptivity of Averaged Stochastic Gradient Descent to Local Strong Convexity for Logistic Regression. J. Mach. Learn. Res. 15 (March): 595-627. 10.1142/S0218194014500065
[13]
Koh K, Kim S, and Boyd S. 2007. An Interior-Point Method for Large-Scale L1-Regularized Logistic Regression. J. Journal of Machine Learning Research. 8 (July 2007): 1519-1555. 10.1109/JSTSP.2007.910971
[14]
Park M, Hastie T. 2007. L1‐Regularization Path Algorithm for Generalized Linear Models. J. Journal of the Royal Statal Society Series B. 69 (August 2007): 659-677. https://doi.org/10.1111/j.1467-9868.2007.00607.x
[15]
Zongben X, Hailiang Guo, Yao Wang, 2012. The Representative of L1/2 Regularization among Lq (1<q≤1) Regularizations: An Experimental Study Based on Phase Diagram. J. Acta Automatica Sinica. 38 (July 2012): 1225-1228 (in Chinese). https://doi.org/10.1016/S1874-1029(11)60293-0
[16]
Zongben X, Xiangyu C, Fengmin X, 2012. L1/2 Regularization: A Thresholding Representation Theory and a Fast Solver. IEEE Transactions on Neural Networks and Learning Systems. United States, 23 (July 2012): 1013-1027. 10.1109/TNNLS.2012.2197412
[17]
Xing Fuchong. 2003. Investigation on Solutions of Cubic Equations with One Unknown. J. Journal of The Central University for Nationalities: Natural Science Edition. 12 (3): 207-218 (in Chinese). 10.3969/j.issn.1005-8036.2003.03.003

Cited By

View all
  • (2024)Prediction of Higher Education Student Dropout based on Regularized Regression ModelsEngineering, Technology & Applied Science Research10.48084/etasr.864414:6(17811-17815)Online publication date: 2-Dec-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICMAI '21: Proceedings of the 2021 6th International Conference on Mathematics and Artificial Intelligence
March 2021
142 pages
ISBN:9781450389464
DOI:10.1145/3460569
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 August 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. L1/2+1 regularization
  2. coordinate descent algorithm
  3. sparse logistic regression
  4. variable selection

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

ICMAI 2021

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)17
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Prediction of Higher Education Student Dropout based on Regularized Regression ModelsEngineering, Technology & Applied Science Research10.48084/etasr.864414:6(17811-17815)Online publication date: 2-Dec-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media