Abstract
Motivated by the advances in the estimation of parameters in linear models by regularization methods such as Ridge and Lasso regularizations, we investigate regularization of Generalized Maximum Entropy, which is an alternative estimation method in linear models. Our simulations confirm the better performance of the regularized Generalized Maximum Entropy estimation method, which could stimulate further theoretical research. An application of the new estimation method is illustrated with data from Thailand concerning the effect of education on economic growth.
This is a preview of subscription content, log in via an institution to check access.
We’re sorry, something doesn't seem to be working properly.
Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.
References
Dudík M, Phillips SJ, Schapire RE (2007) Maximum entropy density estimation with generalized regularization and an application to species distribution modeling. J Mach Learn Res 8:1217–1260
Friedman J, Hastie T, Tibshirani R (2009) Glmnet: Lasso and elastic-net regularized generalized linear models. R Package Version 1(4):1–24
Golan A, Perloff JM (2002) Comparison of maximum entropy and higher-order entropy estimators. J Econ 107(1–2):195–211
Golan A, Judge GG, Miller D (1996) Maximum entropy econometrics: Robust estimation with limited data. Wiley, New York, pp 6–38
Hans C (2009) Bayesian lasso regression. Biometrika 96(4):835–845
Hoerl AE, Kennard RW (1970) Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12(1):55–67
Jaynes ET (1957) Information Theory and Statistical Mechanics II. Phys Rev 108(2):171
Macedo P (2017) Ridge Regression and Generalized Maximum Entropy: An improved version of the Ridge–GME parameter estimator. Commun Stat-Simulation Comput 46(5):3527–3539
Mittelhammer R, Cardell NS, Marsh TL (2013) The data-constrained generalized maximum entropy estimator of the GLM: Asymptotic theory and inference. Entropy 15(5):1756–1775
Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379–423
Tabass MS, Borzadaran GM (2017) The generalized maximum Tsallis entropy estimators and applications to the Portland cement dataset. Commun Stat-Simulation Comput 46(4):3284–3293
Tarkhamtham P, Yamaka W, Sriboonchitta S (2018) The generalize maximum Tsallis entropy estimator in kink regression model. J Phys: Conf Ser 1053(1):012103
Tibshirani R (1996) Regression shrinkage and selection via the lasso. J Roy Stat Soc: Ser B (Methodol) 58(1):267–288
Tsallis C (1988) Possible generalization of Boltzmann-Gibbs statistics. J Stat Phys 52(1):479–487
Wu X (2009) A weighted generalized maximum entropy estimator with a data-driven weight. Entropy 11(4):917–930
Zhao P, Yu B (2006) On model selection consistency of Lasso. J Mach Learn Res 7:2541–2563
Acknowledgment
The author is grateful to two anonymous referees for their valuable suggestions to improve this work. I thank the Centre of Excellence in Econometrics, Chiang Mai University, for financial support.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that there is no conflict of interest.
Additional information
Communicated by Vladik Kreinovich.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Maneejuk, P. On regularization of generalized maximum entropy for linear models. Soft Comput 25, 7867–7875 (2021). https://doi.org/10.1007/s00500-021-05805-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-021-05805-2