Skip to main content
SpringerLink
Log in

Parabolic acceleration of the EM algorithm

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

A new acceleration scheme for optimization procedures is defined through geometric considerations and applied to the EM algorithm. In many cases it is able to circumvent the problem of stagnation. No modification of the original algorithm is required. It is simply used as a software component. Thus the new scheme can be easily implemented to accelerate a fixed point algorithm maximizing some objective function. Some practical examples and simulations are presented to show its ability to accelerate EM-type algorithms converging slowly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Berlinet, A., Roland, Ch.: Acceleration schemes with application to the EM algorithm. Comput. Stat. Data Anal. 51, 3689–3702 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Biernacki, C.: Initializing EM using the properties of its trajectories in Gaussian mixtures. Stat. Comput. 14(3), 267–279 (2004)

    Article  MathSciNet  Google Scholar 

  • Biernacki, C., Celeux, G., Govaert, G.: Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate Gaussian mixture models. Comput. Stat. Data Anal. 41, 561–575 (2003)

    Article  MathSciNet  Google Scholar 

  • Brezinski, C., Zaglia, M.R.: Extrapolation Methods: Theory and Practice. Elsevier Science, Amsterdam (1991)

    MATH  Google Scholar 

  • Cappé, O., Moulines, E., Rydén, T.: Inference in Hidden Markov Models. Springer, New York (2005)

    MATH  Google Scholar 

  • Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. R. Stat. Soc. B 39, 1–38 (1977)

    MATH  MathSciNet  Google Scholar 

  • Hasselblad, V.: Estimation of finite mixtures of distributions from the exponential family. J. Am. Stat. Assoc. 64, 1459–1471 (1969)

    Article  Google Scholar 

  • Jamshidian, M., Jennrich, R.I.: Conjugate gradient acceleration of the EM algorithm. J. Am. Stat. Assoc. 88, 221–228 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  • Jamshidian, M., Jennrich, R.I.: Acceleration of the EM algorithm by using quasi-Newton methods. J. R. Stat. Soc. B 59(3), 569–587 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  • Kent, J.T., Tyler, D.E., Vardi, Y.: A curious likelihood identity for the multivariate t distribution. Commun. Stat. B 23, 441–453 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  • Lange, K.: A quasi-Newton acceleration of the EM algorithm. Stat. Sin. 5, 1–18 (1995)

    MATH  Google Scholar 

  • Lee, E.T.Y.: The rational Bézier representation for conics. In: Farin, G.E. (ed.) Geometric Modeling: Algorithms and New Trends. SIAM, Philadelphia (1987)

    Google Scholar 

  • Liu, C., Rubin, D.B.: The ECME algorithm: a simple extension of EM and ECM with faster monotone convergence. Biometrika 81, 633–648 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  • Liu, C., Rubin, D.B., Wu, Y.N.: Parameter expansion to accelerate EM: The PX-EM algorithm. Biometrika 85, 755–770 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  • Markatou, M., Basu, A., Lindsay, B.G.: Weighted likelihood equations with boostrap root search. J. Am. Stat. Assoc. 93(442), 740–750 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  • McLachlan, G., Krishnan, T.: The EM Algorithm and Extensions, 1st edn. Wiley, New York (1997)

    MATH  Google Scholar 

  • McLachlan, G., Krishnan, T.: The EM Algorithm and Extensions, 2nd edn. Wiley, New York (2008)

    MATH  Google Scholar 

  • McLachlan, G., Peel, D.: Finite Mixture Models. Wiley, New York (2000)

    MATH  Google Scholar 

  • Meng, X.L., Rubin, D.: Maximum likelihood estimation via the ECM algorithm: A general framework. Biometrika 80, 267–278 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  • Meng, X.L., van Dyk, D.: The EM algorithm—an old folk-song sung to a fast new tune. J. R. Stat. Soc. B 59, 511–567 (1997)

    Article  MATH  Google Scholar 

  • Snedecor, G.W., Cochran, W.C.: Statistical Methods. Iowa State University Press, Ames (1967)

    Google Scholar 

  • Titterington, D.M., Smith, A.F.M., Makov, U.E.: Statistical Analysis of Finite Mixture Distributions’ pp. 89–90. Wiley, New York (1985)

    MATH  Google Scholar 

  • Varadhan, R., Roland, Ch.: Squared extrapolation methods (SQUAREM): A new class of simple and efficient numerical schemes for accelerating the convergence of the EM algorithm. Department of Biostatistics Working Paper, Johns Hopkins University (2004). http://www.bepress.com/jhubiostat/paper63

  • Wolfe, J.H.: Pattern clustering by multivariate mixture analysis. Multivar. Behav. Res. 5, 329–350 (1970)

    Article  Google Scholar 

  • Wu, C.F.J.: On the convergence properties of the EM algorithm. Ann. Stat. 11, 95–103 (1983)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut de Mathématiques et de Modélisation de Montpellier, UMR CNRS 5149, Equipe de Probabilités et Statistique, Université Montpellier II, Cc 051, Place Eugène Bataillon, 34095, Montpellier Cedex 5, France

    A. Berlinet

  2. Laboratoire Paul Painlevé, UFR de Mathématiques Pures et Appliquées, Université des Sciences et Technologies de Lille, Cité Scientifique, 59655, Villeneuve d’Ascq cedex, France

    C. Roland

Authors
  1. A. Berlinet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C. Roland
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to C. Roland.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Berlinet, A., Roland, C. Parabolic acceleration of the EM algorithm. Stat Comput 19, 35–47 (2009). https://doi.org/10.1007/s11222-008-9067-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-008-9067-x

Keywords

Search

Navigation