Skip to main content
SpringerLink
Log in

Analysis of generalized linear mixed models via a stochastic approximation algorithm with Markov chain Monte-Carlo method

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

In recent years much effort has been devoted to maximum likelihood estimation of generalized linear mixed models. Most of the existing methods use the EM algorithm, with various techniques in handling the intractable E-step. In this paper, a new implementation of a stochastic approximation algorithm with Markov chain Monte Carlo method is investigated. The proposed algorithm is computationally straightforward and its convergence is guaranteed. A simulation and three real data sets, including the challenging salamander data, are used to illustrate the procedure and to compare it with some existing methods. The results indicate that the proposed algorithm is an attractive alternative for problems with a large number of random effects or with high dimensional intractable integrals in the likelihood function.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aitkin M. 1996. A general maximum likelihood analysis of overdispersion in generalized linear models. Statistics and Computing 6: 251-262.

    Google Scholar 

  • Booth J.G. and Hobert J.P. 1999. Maximum generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm. Journal of Royal Statistical Society, Series B 61: 265-285.

    Google Scholar 

  • Breslow N.E. 1984. Extra-Possion variation in log-linear models. Applied Statistics 33: 38-44.

    Google Scholar 

  • Breslow N.E. and Clayton D.G. 1993. Approximate inference in generalized linear mixed models. Journal of the American Statistical Association 88: 9-25.

    Google Scholar 

  • Crowder M.J. 1978. Beta-Binomial ANOVA for proportions. Applied Statistics 27: 34-37.

    Google Scholar 

  • Cox D.R. and Snell, E.J. 1988. Analysis of Binary Data, 2nd edn. Chapman and Hall, London.

    Google Scholar 

  • Dempster A.P., Laird N.M., and Rubin D.B. 1977. Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of Royal Statistical Society, Series B 39: 1-38.

    Google Scholar 

  • Dorn H.F. 1954. The relationship of cancer of the lung and the use of tobacco. American Statistician 8: 7-13.

    Google Scholar 

  • Gamerman D. 1997. Sampling from the posterior distribution in generalized linear mixed models. Statistics and Computing 7: 57-68.

    Google Scholar 

  • Geman S. and Geman D. 1984. Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 6: 721-741.

    Google Scholar 

  • Gu M.G. and Kong F.H. 1998. A stochastic approximation algorithm with Markov chain Monte-Carlo method for incomplete data estimation problems. Proceeding of National Academy Sciences, USA 95: 7270-7274.

    Google Scholar 

  • Gu M.G. and Li S.L. 1998. A stochastic approximation algorithm for maximum-likelihood estimation with incomplete data. The Canadian Journal of Statistics 26: 567-582

    Google Scholar 

  • Hastings W.K. 1970. Monte Carlo sampling methods using Markov chains and their application. Biometrika 57: 97-109.

    Google Scholar 

  • Jiang J.M. 1998. Consistent estimators in generalized linear mixed models. Journal of the American Statistical Association 93: 720-729.

    Google Scholar 

  • Karim M.R. and Zeger S.L. 1992. Generalized linear models with random effects; salamander mating revisited. Biometrics 48: 631-644.

    Google Scholar 

  • Kushner H.J. and Yin G.G. 1997. Stochastic Approximation Algorithms and Applications. Springer, New York.

    Google Scholar 

  • Kuk A.Y.C. and Cheng Y.W. 1999. Pointwise and functional approximations in Monte Carlo maximum likelihood estimation. Statistics and Computing 9: 91-99.

    Google Scholar 

  • Lee Y. and Nelder J.A. 1996. Hierarchical generalized linear models (with discussion). Journal of Royal Statistical Society B, Series B 58: 619-678.

    Google Scholar 

  • Liu Q. and Pierce D.A. 1993. Heterogeneity in Mantel-Haenszel-type models. Biometrika 80: 543-556.

    Google Scholar 

  • Manton K.G., Woodbury M.A., Stallard E., Riggan W.B., Creason J.P., and Pellom A.C. 1989. Empirical Bayes procedures for stabilizing maps of U.S. cancer mortality rates. Journal of the American Statistical Association 84: 637-650.

    Google Scholar 

  • McCullagh P. and Nelder J.A. 1989. Generalized Linear Models, 2nd edn. Chapman and Hall, London.

    Google Scholar 

  • McCulloch C.E. 1994. Maximum likelihood variance components estimation for binary data. Journal of American Statistical Association 89: 330-335.

    Google Scholar 

  • McCulloch C.E. 1997. Maximum likelihood algorithms for generalized linear mixed models. Journal of the American Statistical Association 92: 162-170.

    Google Scholar 

  • McGilchrist C.A. 1994. Estimation in generalized linear mixed models. Journal of Royal Statistical Society, Series B 56: 61-69.

    Google Scholar 

  • Metropolis N., Rosenbluth A.W., Rosenbluth M.N., Teller A.H., and Teller E. 1953. Equations of state calculations by fast computing machine. Journal of Chemical Physics 21: 1087-1091.

    Google Scholar 

  • Quintana F.A., Liu J.S., and del Pino G.E. 1999. Monte Carlo EM with importance reweighting and its applications in random effect models. Computational Statistics & Data Analysis 29: 429-444.

    Google Scholar 

  • Robbins H. and Monro S. 1951. A stochastic approximation method. Ann. Math. Statist. 22: 400-407.

    Google Scholar 

  • Schall R. 1991. Estimation in generalized linear mixed models. Biometrika 78: 719-727.

    Google Scholar 

  • Shun Z.M. 1997. Another look at the salamander mating data: A modified Laplace approximation approach. Journal of the American Statistical Association 92: 341-349.

    Google Scholar 

  • Solomon P.J. and Cox D.R. 1992. Nonlinear components of variance models. Biometrika 79: 1-11.

    Google Scholar 

  • Spiegelhalter D.J., Thomas A., Best N., and Gilks W.R. 1993. BUGS Example 0.30, MRC Biostatistics Unit, Cambridge.

    Google Scholar 

  • Steele B.M. 1996. A modified EM algorithm for estimation in general-ized mixed models. Biometrics 52: 1295-1310.

    Google Scholar 

  • Stiratelli R., Laird N., and Ware J. 1984. Random effects models for serial observations with binary responses. Biometrics 40: 961-971.

    Google Scholar 

  • Tanner M.A. 1996. Tools for Statistical Inference: Methods for the Exploration of Posterior Distributions and Likelihood Functions, 2nd edn. Springer-Verlag, New York.

    Google Scholar 

  • Thall P.F. and Vail S.C. 1990. Some covariance models for longitudinal count data with overdispersion. Biometrics 46: 657-671.

    Google Scholar 

  • Williams D.A. 1982. Extra-Binomial variation in logistic linear models. Applied Statistics 31: 144-148.

    Google Scholar 

  • Zeger S.L. and Karim M.R. 1991. Generalized linear models with random effects: A Gibbs sampling approach. Journal of the American Statistical Association 86: 79-86.

    Google Scholar 

  • Zeger S.L., Liang K.Y., and Albert P.S. 1988. Models for longitudinal data: A generalized estimating equation approach. Biometrics 44: 1049-1060.

    Google Scholar 

Download references

Authors
  1. Hong-Tu Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sik-Yum Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhu, HT., Lee, SY. Analysis of generalized linear mixed models via a stochastic approximation algorithm with Markov chain Monte-Carlo method. Statistics and Computing 12, 175–183 (2002). https://doi.org/10.1023/A:1014890720461

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1014890720461

Search

Navigation