Abstract
The discriminant method, which uses a basis expansion in the logistic regression model and estimates it by a simply regularized likelihood, is considerably efficient especially when the discrimination boundary is complex. However, when the complexities of the boundary are different by region, the method tends to cause under-fitting or/and over-fitting at some regions. To overcome this difficulty, a smoothly varying regularization is proposed in the framework of the logistic regression. Through simulation studies based on synthetic data, the superiority of the proposed method to some existing methods is checked.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)
Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96, 1348–1360 (2001). https://doi.org/10.1198/016214501753382273
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning, 2nd edn. Springer, New York (2009)
Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12, 55–67 (1970). https://doi.org/10.1080/00401706.1970.10488634
Kim, D., Kawano, S., Ninomiya, Y. Smoothly varying ridge regularization (2021). arXiv preprint, 2102.00136
Stone, M.: Cross validatory choice and assessment of statistical predictions. J. R. Stat. Soc. Ser. B 36, 111–133 (1974). https://doi.org/10.1111/j.2517-6161.1974.tb00994.x
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B 58, 267–288 (1996). https://doi.org/10.1111/j.2517-6161.1996.tb02080.x
Tibshirani, R., Saunders, M., Rosset, S.: Sparsity and smoothness via the fused lasso. J. R. Stat. Soc. Ser. B 67, 91–108 (2005). https://doi.org/10.1111/j.1467-9868.2005.00490.x
Tibshirani, R.J., Taylor, J.: The solution path of the generalized lasso. Ann. Statist. 39, 1335–1371 (2011). https://doi.org/10.1214/11-AOS878
Zhang, C.H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Statist. 38, 894–942 (2010). https://doi.org/10.1214/09-AOS729
Zou, H.: The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101, 1418–1429 (2006). https://doi.org/10.1198/016214506000000735
Acknowledgements
We are grateful to the editors and two referees for their helpful comments. SK was supported by JSPS KAKENHI Grant Numbers JP19K11854 and JP20H02227, and YN was supported by JSPS KAKENHI Grant Numbers JP16K00050.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yoshida, H., Kawano, S., Ninomiya, Y. (2021). Discriminant Analysis via Smoothly Varying Regularization. In: Czarnowski, I., Howlett, R.J., Jain, L.C. (eds) Intelligent Decision Technologies. Smart Innovation, Systems and Technologies, vol 238. Springer, Singapore. https://doi.org/10.1007/978-981-16-2765-1_37
Download citation
DOI: https://doi.org/10.1007/978-981-16-2765-1_37
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-2764-4
Online ISBN: 978-981-16-2765-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)