Abstract
Three general algorithms that use different strategies are proposed for computing the maximum likelihood estimate of a semiparametric mixture model. They seek to maximize the likelihood function by, respectively, alternating the parameters, profiling the likelihood and modifying the support set. All three algorithms make a direct use of the recently proposed fast and stable constrained Newton method for computing the nonparametric maximum likelihood of a mixing distribution and employ additionally an optimization algorithm for unconstrained problems. The performance of the algorithms is numerically investigated and compared for solving the Neyman-Scott problem, overcoming overdispersion in logistic regression models and fitting two-level mixed effects logistic regression models. Satisfactory results have been obtained.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aitkin, M.: A general maximum likelihood analysis of overdispersion in generalised linear models. Stat. Comput. 6, 251–262 (1996)
Aitkin, M.: A general maximum likelihood analysis of variance components in generalized linear models. Biometrics 55, 117–128 (1999)
Böhning, D.: A review of reliable algorithms for the semi-parametric maximum likelihood estimator of a mixture distribution. J. Stat. Plan. Inference 47, 5–28 (1995)
Böhning, D.: The EM algorithm with gradient function update for discrete mixtures with known (fixed) number of components. Stat. Comput. 13, 257–265 (2003)
Dax, A.: The smallest point of a polytope. J. Optim. Theory Appl. 64, 429–432 (1990)
DerSimonian, R.: Maximum likelihood estimation of a mixing distribution. J. R. Stat. Soc., Ser. C 35, 302–309 (1986)
Einbeck, J., Darnell, R., Hinde, J.: NPMLREG: Nonparametric maximum likelihood estimation for random effect models. R Package Version 0.43 (2007)
Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley, New York (1987)
Follmann, D.A., Lambert, D.: Generalizing logistic regression by nonparametric mixing. J. Am. Stat. Assoc. 84, 295–300 (1989)
Haskell, K.H., Hanson, R.J.: An algorithm for linear least squares problems with equality and nonnegativity constraints. Math. Program. 21, 98–118 (1981)
Heckman, J., Singer, B.: A method for minimizing the impact of distributions assumptions in econometric models for duration data. Econometrica 52, 271–320 (1984)
Jamshidian, M., Jennrich, R.I.: Acceleration of the EM algorithm by using quasi-Newton methods. J. R. Stat. Soc., Ser. B 59, 569–587 (1997)
Kiefer, J., Wolfowitz, J.: Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters. Ann. Math. Stat. 27, 886–906 (1956)
Laird, N.M.: Nonparametric maximum likelihood estimation of a mixing distribution. J. Am. Stat. Assoc. 73, 805–811 (1978)
Lawson, C.L., Hanson, R.J.: Solving Least Squares Problems. Prentice-Hall, New York (1974)
Lindsay, B.G.: The geometry of mixture likelihoods: A general theory. Ann. Stat. 11, 86–94 (1983)
Lindsay, B.G.: Mixture Models: Theory, Geometry and Applications. NSF-CBMS Regional Conference Series in Probability and Statistics, vol. 5. Institute for Mathematical Statistics, Hayward (1995)
Lindsay, B.G., Lesperance, M.L.: A review of semiparametric mixture models. J. Stat. Plan. Inference 47, 29–39 (1995)
Murphy, S.A., van der Vaart, A.W.: On profile likelihood. J. Am. Stat. Assoc. 95, 449–465 (2000)
Neyman, J., Scott, E.L.: Consistent estimates based on partially consistent observations. Econometrica 16, 1–32 (1948)
Powell, M.J.D.: On search directions for minimization algorithms. Math. Program. 4, 193–201 (1973)
R Development Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (2006)
Tsodikov, A.: Semiparametric models: a generalized self-consistency approach. J. R. Stat. Soc., Ser. B 65, 759–774 (2003)
Wang, Y.: On fast computation of the non-parametric maximum likelihood estimate of a mixing distribution. J. R. Stat. Soc., Ser. B 69, 185–198 (2007)
Wang, Y.: Dimension-reduced nonparametric maximum likelihood computation for interval-censored data. Comput. Stat. Data Anal. 52, 2388–2402 (2008)
Zeng, D., Lin, D.Y.: Maximum likelihood estimation in semiparametric regression models with censored data. J. R. Stat. Soc., Ser. B 69, 507–564 (2007)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, Y. Maximum likelihood computation for fitting semiparametric mixture models. Stat Comput 20, 75–86 (2010). https://doi.org/10.1007/s11222-009-9117-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-009-9117-z