Skip to main content
SpringerLink
Log in

Expectation Propagation

  • Reference work entry
  • First Online:
Encyclopedia of Machine Learning and Data Mining

  • 159 Accesses

Synonyms

EP

Definition

Expectation propagation is an algorithm for Bayesian machine learning. It tunes the parameters of a simpler approximate distribution (e.g., a Gaussian) to match the exact posterior distribution of the model parameters given the data. Expectation propagation operates by propagating messages, similar to the messages in (loopy) belief propagation. Whereas messages in belief propagation correspond to exact belief states, messages in expectation propagation correspond to approximations of the belief states in terms of expectations, such as means and variances. It is a deterministic method especially well suited to large databases and dynamic systems, where exact methods for Bayesian inference fail and Monte Carlo methods are far too slow.

Motivation and Background

One of the main problems for Bayesian methodsis their computational expense: computation of the exact posterior given the observed data typically requires the solution of high-dimensional integrals that...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 699.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 949.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Recommended Reading

  • Barthelmé S, Chopin N (2014) Expectation propagation for likelihood-free inference. J Am Stat Assoc 109(505):315–333

    Article  MathSciNet  Google Scholar 

  • Csató L (2002) Gaussian processes – iterative sparse approximations. Ph.D. thesis, Aston Uni-versity

    Google Scholar 

  • Cseke B, Heskes TT (2011) Approximate marginals in latent Gaussian models. J Mach Learn Res 12:417–454

    MathSciNet  MATH  Google Scholar 

  • Herbrich R, Graepel T (2006) TrueSkill: a Bayesian skill rating system. Technical report (MSR-TR-2006-80), Microsoft Research, Cambridge

    Google Scholar 

  • Heskes T, Zoeter O (2002) Expectation propagation for approximate inference in dynamic Bayesian networks. In: Darwiche A, Friedman N (eds) Proceedings of the 18th conference on uncertainty in artificial intelligence, Alberta, pp 216–223

    Google Scholar 

  • Heskes T, Opper M, Wiegerinck W, Winther O, Zoeter O (2005) Approximate inference with expectation constraints. J Stat Mech Theory Exp P11015

    Google Scholar 

  • Minka T (2001) A family of algorithms for approximate Bayesian inference. Ph.D. thesis, MIT

    Google Scholar 

  • Minka T (2005) Divergence measures and message passing. Technical report (MSR-TR-2005-173), Microsoft Research, Cambridge

    Google Scholar 

  • Minka T, Lafferty J (2002) Expectation-propagation for the generative aspect model. In: Darwiche A, Friedman N (eds) Proceedings of the 18th conference on uncertainty in artificial intelligence, Alberta, pp 352–359

    Google Scholar 

  • Opper M, Paquet U, Winther O (2013) Perturbative corrections for approximate inference in Gaussian latent variable models. J Mach Learn Res 14(1):2857–2898

    MathSciNet  MATH  Google Scholar 

  • Opper M, Winther O (2001) Tractable approximations for probabilistic models: the adaptive Thouless-Anderson-Palmer mean field approach. Phys Rev Lett 86:3695–3699

    Article  Google Scholar 

  • Seeger M (2008) Bayesian inference and optimal design for the sparse linear model. J Mach Learn Res 9: 759–813

    MathSciNet  MATH  Google Scholar 

  • Seeger M, Nickisch H (2010) Fast convergent algorithms for expectation propagation approximate Bayesian inference. arXiv preprint arXiv:1012.3584

    Google Scholar 

  • Welling M, Minka T, Teh Y (2005) Structured region graphs: morphing EP into GBP. In: Bacchus F, Jaakkola T (eds) Proceedings of the 21st conference on uncertainty in artificial intelligence (UAI), Edinburgh, pp 609

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Computing and Information Sciences, Radboud University Nijmegen, Heyendaalseweg 135, 6525, Nijmegen, The Netherlands

    Tom Heskes

Authors
  1. Tom Heskes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tom Heskes .

Editor information

Editors and Affiliations

  1. The University of New South Wales, Sydney, NSW, Australia

    Claude Sammut

  2. Faculty of Information Technology, Monash University, Melbourne, VIC, Australia

    Geoffrey I. Webb

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Science+Business Media New York

About this entry

Cite this entry

Heskes, T. (2017). Expectation Propagation. In: Sammut, C., Webb, G.I. (eds) Encyclopedia of Machine Learning and Data Mining. Springer, Boston, MA. https://doi.org/10.1007/978-1-4899-7687-1_95

Download citation

Publish with us

Policies and ethics

Search

Navigation