Abstract
One of the estimating equations of the Maximum Likelihood Estimation method, for finite mixtures of the one parameter exponential family, is the first moment equation. This can help considerably in reducing the labor and the cost of calculating the Maximum Likelihood estimates. In this paper it is shown that the EM algorithm can be substantially improved by using this result when applied for mixture models. A short discussion about other methods proposed for the calculation of the Maximum Likelihood estimates are also reported showing that the above findings can help in this direction too.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aitkin, M. and Aitkin, I. (1996) An hybrid EM/Gauss-Newton algorithm for maximum likelihood in mixture distributions. Statistics and Computing, 6, 127–130.
Behboodian, J. (1970) On a mixture of normal distributions. Biometrika, 56, 215–217.
Bohning, D. (1995) A review of reliable maximum likelihood algorithms for semiparametric mixture models. Journal of Statistical Planning and Inference, 47, 5–28.
Bohning, D., Dietz, Ek., Schaub, R., Schlattman, P. and Lindsay, B. (1994) The distribution of the likelihood ratio for mixtures of densities from the one-parameter exponential family. Annals of the Institute of Statistical Mathematics, 46, 373–388.
Day, N. E. (1969) Estimating the components of a mixture of normal distributions. Biometrika, 56, 463–474.
Dempster, A. P., Laird, N. M. and Rubin, D. (1977) Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society,B39, 1–38.
Fruman, W. D. and Lindsay, B. (1994) Measuring the relative effectiveness of moment estimators as starting values in maximizing mixture likelihoods. Computational Statistics and Data Analysis, 17, 493–507.
Hasselblad, V. (1969) Estimation of finite mixtures from the exponential family. Journal of the American Statistical Association, 64, 1459–1471.
Johnson, N., Kotz, S. and Kemp, A. (1992). Univariate discrete distributions (2nd edition). Willey, New York.
Karlis, D. and Xekalaki, E. (1998) Minimum Hellinger distance Estimation for finite Poisson mixtures. Computational Statistics and Data Analysis, 29, 81–103.
Kemp, A. W. (1986) Weighted discrepancies and Maximum Likelihood for discrete distributions. Communications in Statistics, 15, 783–801.
Lange, K. (1995) A Quasi-Newton Acceleration of the EM algorithm. Statistica Sinica, 5, 1–8.
Lindsay, B. (1981) Properties of the Maximum Likelihood Estimator of a Mixing Distribution. In G. P. Patil (ed) Statistical Distributions in Scientific Work, 5, pp 95–109, Reidel, Boston.
Lindsay, B. (1983) The geometry of mixture likelihood. A general theory. Annals of Statistics, 11, 86–94.
Lindsay, B. (1995) Mixture Models: Theory, Geometry and Applications.Regional Conference Series in Probability and Statistics, Vol 5, Institute of Mathematical Statistics and American Statistical Association.
Lindsay, B. and Roeder, K. (1995) A review of semiparametric mixture models. Journal of Statistical Planning and Inference, 47, 29–39.
McLachlan, G. and Krishnan, T. (1997) The EM Algorithm and Extensions. Willey Series.
Sprott, D. (1983) Estimating the parameters of a convolution by Maximum Likelihood. Journal of the American Statistical Association, 78, 457–460.
Rights and permissions
About this article
Cite this article
Karlis, D., Xekalaki, E. Improving the EM algorithm for mixtures. Statistics and Computing 9, 303–307 (1999). https://doi.org/10.1023/A:1008968107680
Issue Date:
DOI: https://doi.org/10.1023/A:1008968107680