Abstract
We consider the use of Monte Carlo methods to obtain maximum likelihood estimates for random effects models and distinguish between the pointwise and functional approaches. We explore the relationship between the two approaches and compare them with the EM algorithm. The functional approach is more ambitious but the approximation is local in nature which we demonstrate graphically using two simple examples. A remedy is to obtain successively better approximations of the relative likelihood function near the true maximum likelihood estimate. To save computing time, we use only one Newton iteration to approximate the maximiser of each Monte Carlo likelihood and show that this is equivalent to the pointwise approach. The procedure is applied to fit a latent process model to a set of polio incidence data. The paper ends by a comparison between the marginal likelihood and the recently proposed hierarchical likelihood which avoids integration altogether.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Anderson, D. A. and Aitkin, M. A. (1985) Variance component models with binary response: interviewer variability. J. R. Statist. Soc. B, 47, 203–210.
Breslow, N. E. and Clayton, D. G. (1993) Approximate inference in generalized linear mixed models. J. Am. Statist. Ass., 88, 9–25.
Chan, K. S. and Ledolter, J. (1995) Monte Carlo EM estimation for time series models involving counts. J. Amer. Statist. Assoc., 90, 242–252.
Crouch, E. A. C. and Spiegelman, D. (1990) The evaluation of integrals of the from ∫ f(t)exp(–t 2)dt: application to logistic-normal models. J. Am. Statist. Ass., 85, 464–469.
Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977) Maxi-mum likelihood from incomplete data via the EM algorithm (with discussion). J. R. Statist. Soc. B, 39, 1–38.
Geyer, C. J. (1994) On the convergence of Monte Carlo maxi-mum likelihood calculations. J. R. Statist. Soc. B., 56, 261–274.
Geyer, C. J. and Thompson, E. A. (1992) Constrained maximum likelihood for dependent data (with discussion). J. R. Statist. Soc. B., 54, 657–699.
Karim, M. R. and Zeger, S. L. (1992) Generalized linear models with random effects; salamander mating revisited. Biometrics, 48, 631–644.
Kleinman, J. C. (1973) Proportions with extraneous variance: single and independent samples. J. Amer. Statist. Assoc., 68, 46–54.
Kuk, A. Y. C. and Cheng, Y. W. (1997) The Monte Carlo Newton-Raphson algorithm. J. Statist. Comput. Simul., 59, 233–250.
Lange, K. (1995) A gradient algorithm locally equivalent to the EM algorithm. J. R. Statist. Soc. B, 57, 425–437.
Lee, Y. and Nelder, J. A. (1996) Hierarchical generalized linear models (with discussion). J. R. Statist. Soc. B, 58, 619–678.
Louis, T. A. (1982) Finding the observed information matrix when using the EM algorithm. J. R. Statist. Soc. B, 44, 226–233.
McCulloch, C. E. (1997) Maximum likelihood algorithms for generalized linear mixed models. J. Am. Statist. Assoc., 92, 162–170.
McGilchrist, C. A. (1994) Estimation in generalized mixed mod-els. J. R. Statist. Soc. B, 56, 61–69.
McGilchrist, C. A. and Aisbett, C. W. (1991a) Restricted BLUP for mixed linear models. Biometr. J., 32, 545–550.
McGilchrist, C. A. and Aisbett, C. W. (1991b) Regression with frailty in survival analysis. Biometrics, 47, 461–466.
Penttinen, A. (1984) Modelling interaction in spatial point pat-terns: parameter estimation by the maximum likelihood method. Jy. Stud. Comput. Sci. Econ. Statist., 7, 1–105
Schall, R. (1991) Estimation in generalised linear models with random effects. Biometrika, 78, 719–727.
Smith, D. M. (1983) Maximum likelihood estimation of the pa-rameters of the beta binomial distribution. Applied Statist., 32, 196–204.
Wei, G. C. G. and Tanner, M. A. (1990) A Monte Carlo imple-mentation of the EM algorithm and the poor man's data augmentation algorithms. J. Amer. Statist. Assoc., 85, 699–704.
Weil, C. S. (1970) Selection of the valid number of sampling units and consideration of their combination in toxicological studies involving reproduction, teratogenesis or carcinogen-esis. Food and Cosmetic Toxicology, 8, 177–182.
Zeger, S. L. (1988) A regression model for time series of counts. Biometrika, 75, 621–629.
Rights and permissions
About this article
Cite this article
Kuk, A.Y.C., Cheng, Y.W. Pointwise and functional approximations in Monte Carlo maximum likelihood estimation. Statistics and Computing 9, 91–99 (1999). https://doi.org/10.1023/A:1008800715000
Issue Date:
DOI: https://doi.org/10.1023/A:1008800715000