Skip to main content
SpringerLink
Log in

Pointwise and functional approximations in Monte Carlo maximum likelihood estimation

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

We consider the use of Monte Carlo methods to obtain maximum likelihood estimates for random effects models and distinguish between the pointwise and functional approaches. We explore the relationship between the two approaches and compare them with the EM algorithm. The functional approach is more ambitious but the approximation is local in nature which we demonstrate graphically using two simple examples. A remedy is to obtain successively better approximations of the relative likelihood function near the true maximum likelihood estimate. To save computing time, we use only one Newton iteration to approximate the maximiser of each Monte Carlo likelihood and show that this is equivalent to the pointwise approach. The procedure is applied to fit a latent process model to a set of polio incidence data. The paper ends by a comparison between the marginal likelihood and the recently proposed hierarchical likelihood which avoids integration altogether.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Anderson, D. A. and Aitkin, M. A. (1985) Variance component models with binary response: interviewer variability. J. R. Statist. Soc. B, 47, 203–210.

    Google Scholar 

  • Breslow, N. E. and Clayton, D. G. (1993) Approximate inference in generalized linear mixed models. J. Am. Statist. Ass., 88, 9–25.

    Google Scholar 

  • Chan, K. S. and Ledolter, J. (1995) Monte Carlo EM estimation for time series models involving counts. J. Amer. Statist. Assoc., 90, 242–252.

    Google Scholar 

  • Crouch, E. A. C. and Spiegelman, D. (1990) The evaluation of integrals of the from ∫ f(t)exp(–t 2)dt: application to logistic-normal models. J. Am. Statist. Ass., 85, 464–469.

    Google Scholar 

  • Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977) Maxi-mum likelihood from incomplete data via the EM algorithm (with discussion). J. R. Statist. Soc. B, 39, 1–38.

    Google Scholar 

  • Geyer, C. J. (1994) On the convergence of Monte Carlo maxi-mum likelihood calculations. J. R. Statist. Soc. B., 56, 261–274.

    Google Scholar 

  • Geyer, C. J. and Thompson, E. A. (1992) Constrained maximum likelihood for dependent data (with discussion). J. R. Statist. Soc. B., 54, 657–699.

    Google Scholar 

  • Karim, M. R. and Zeger, S. L. (1992) Generalized linear models with random effects; salamander mating revisited. Biometrics, 48, 631–644.

    Google Scholar 

  • Kleinman, J. C. (1973) Proportions with extraneous variance: single and independent samples. J. Amer. Statist. Assoc., 68, 46–54.

    Google Scholar 

  • Kuk, A. Y. C. and Cheng, Y. W. (1997) The Monte Carlo Newton-Raphson algorithm. J. Statist. Comput. Simul., 59, 233–250.

    Google Scholar 

  • Lange, K. (1995) A gradient algorithm locally equivalent to the EM algorithm. J. R. Statist. Soc. B, 57, 425–437.

    Google Scholar 

  • Lee, Y. and Nelder, J. A. (1996) Hierarchical generalized linear models (with discussion). J. R. Statist. Soc. B, 58, 619–678.

    Google Scholar 

  • Louis, T. A. (1982) Finding the observed information matrix when using the EM algorithm. J. R. Statist. Soc. B, 44, 226–233.

    Google Scholar 

  • McCulloch, C. E. (1997) Maximum likelihood algorithms for generalized linear mixed models. J. Am. Statist. Assoc., 92, 162–170.

    Google Scholar 

  • McGilchrist, C. A. (1994) Estimation in generalized mixed mod-els. J. R. Statist. Soc. B, 56, 61–69.

    Google Scholar 

  • McGilchrist, C. A. and Aisbett, C. W. (1991a) Restricted BLUP for mixed linear models. Biometr. J., 32, 545–550.

    Google Scholar 

  • McGilchrist, C. A. and Aisbett, C. W. (1991b) Regression with frailty in survival analysis. Biometrics, 47, 461–466.

    Google Scholar 

  • Penttinen, A. (1984) Modelling interaction in spatial point pat-terns: parameter estimation by the maximum likelihood method. Jy. Stud. Comput. Sci. Econ. Statist., 7, 1–105

    Google Scholar 

  • Schall, R. (1991) Estimation in generalised linear models with random effects. Biometrika, 78, 719–727.

    Google Scholar 

  • Smith, D. M. (1983) Maximum likelihood estimation of the pa-rameters of the beta binomial distribution. Applied Statist., 32, 196–204.

    Google Scholar 

  • Wei, G. C. G. and Tanner, M. A. (1990) A Monte Carlo imple-mentation of the EM algorithm and the poor man's data augmentation algorithms. J. Amer. Statist. Assoc., 85, 699–704.

    Google Scholar 

  • Weil, C. S. (1970) Selection of the valid number of sampling units and consideration of their combination in toxicological studies involving reproduction, teratogenesis or carcinogen-esis. Food and Cosmetic Toxicology, 8, 177–182.

    Google Scholar 

  • Zeger, S. L. (1988) A regression model for time series of counts. Biometrika, 75, 621–629.

    Google Scholar 

Download references

Authors
  1. Anthony Y. C. Kuk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuk W. Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kuk, A.Y.C., Cheng, Y.W. Pointwise and functional approximations in Monte Carlo maximum likelihood estimation. Statistics and Computing 9, 91–99 (1999). https://doi.org/10.1023/A:1008800715000

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008800715000

Search

Navigation