Skip to main content
Springer Nature Link
Log in

A parameter expansion version of the SAEM algorithm

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

The EM algorithm and its extensions are very popular tools for maximum likelihood estimation in incomplete data setting. However, one of the limitations of these methods is their slow convergence. The PX-EM (parameter-expanded EM) algorithm was proposed by Liu, Rubin and Wu to make EM much faster. On the other hand, stochastic versions of EM are powerful alternatives of EM when the E-step is untractable in a closed form. In this paper we propose the PX-SAEM which is a parameter expansion version of the so-called SAEM (Stochastic Approximation version of EM). PX-SAEM is shown to accelerate SAEM and improve convergence toward the maximum likelihood estimate in a parametric framework. Numerical examples illustrate the behavior of PX-SAEM in linear and nonlinear mixed effects models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Celeux G. and Diebolt J. 1985. The SEM algorithm: A probabilistic teacher algorithm derived from the EM algorithm for the mixture problem. Computational. Statistics Quaterly 2: 73–82.

    Google Scholar 

  • Concordet D. and Nunez O. 2002. A simulated pseudo–maximum likelihood estimator for nonlinear mixed models. Comput. Statist. Data Anal. 39: 187–201.

    Article  MATH  MathSciNet  Google Scholar 

  • Delyon B., Lavielle M., and Moulines E. 1999. Convergence of a stochastic approximation version of the EM algorithm. Ann. Statist. 27: 94–128.

    Article  MATH  MathSciNet  Google Scholar 

  • Dempster A., Laird N., and Rubin D. 1977. Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Statist. Soc. Ser. B 39: 1–38. With discussion.

    MATH  MathSciNet  Google Scholar 

  • Fesslern J. and Hero A. 1994. Space-alternating generalized expectation-maximization algorithm. IEE Trans. Sig. Proc. 42: 2664–2677.

    Article  Google Scholar 

  • Jank W. 2004. Quasi–Monte Carlo sampling to improve the efficiency of Monte Carlo EM. CSDA 48: 685–701.

    MathSciNet  Google Scholar 

  • Kuhn E. and Lavielle M. 2004. Coupling a stochastic approximation version of EM with a MCMC procedure. ESAIM P&S 8: 115–131.

    Article  MATH  MathSciNet  Google Scholar 

  • Kuhn E. and Lavielle M. 2005. Maximum likelihood estimation in nonlinear mixed effects models. CSDA 49(4), 20–38.

    MathSciNet  Google Scholar 

  • Liu C. and Rubin D. 1994. The ECME algorithm: a simple extension of em and ecm with faster monotone convergence. Biometrika 81: 633–648.

    Article  MATH  MathSciNet  Google Scholar 

  • Liu C., Rubin D., and Wu Y. 1998. Parameter expansion to accelerate EM: The PX–EM algorithm. Biometrika 85: 755–770.

    Article  MATH  MathSciNet  Google Scholar 

  • McLachlan G. and Krishnan T. 1997. The EM Algorithm and Extensions. New York: J. Wiley and Sons.

    MATH  Google Scholar 

  • Meng X.-L. and Rubin D.B. 1993. Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika 80(2): 267–278.

    Article  MATH  MathSciNet  Google Scholar 

  • Wei G. and Tanner M. 1990. A Monte Carlo implementation of the EM algorithm and the poor man’s data augmentation algorithms. J. Amer. Statist. Assoc. 85: 699–704.

    Article  Google Scholar 

  • Wu C. 1983. On the convergence properties of the EM algorithm. Ann. Statist. 11: 95–103.

    MATH  MathSciNet  Google Scholar 

  • Wu L. 2004. Exact and approximate inferences for nonlinear mixed-effects models with missing covariates. J. Amer. Statist. Assoc. 99: 700–709.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. INRIA Futurs and Université Paris-Sud, Equipe de Probabilités, Statistique et Modélisation, Bat 425, 91400, Orsay, France

    Marc Lavielle

  2. Laboratoire de Mathématiques, Université Paris-Sud, 91405, Orsay Cedex, France

    Cristian Meza

Authors
  1. Marc Lavielle
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cristian Meza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marc Lavielle.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lavielle, M., Meza, C. A parameter expansion version of the SAEM algorithm. Stat Comput 17, 121–130 (2007). https://doi.org/10.1007/s11222-006-9007-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-006-9007-6

Keywords

Search

Navigation