Expectation Propagation

Synonyms

EP

Definition

Expectation propagation is an algorithm for Bayesian machine learning (see Bayesian Methods). It tunes the parameters of a simpler approximate distribution (e.g., a Gaussian) to match the exact posterior distribution of the model parameters given the data. Expectation propagation operates by propagating messages, similar to the messages in (loopy) belief propagation (see Graphical Models). Whereas messages in belief propagation correspond to exact belief states, messages in expectation propagation correspond to approximations of the belief states in terms of expectations, such as means and variances. It is a deterministic method especially well-suited to large databases and dynamic systems, where exact methods for Bayesian inference fail and Monte Carlo methods are far too slow.

Motivation and Background

One of the main problems for Bayesian methodsare their computational expense: computation of the exact posterior, given the observed data, typically requires...