Bayes Rule

Definition

Bayes rule provides a decomposition of a conditional probability that is frequently used in a family of learning techniques collectively called Bayesian Learning. Bayes rule is the equality

$$\mathrm{P}(z\,\vert \,w) = \frac{\mathrm{P}(z)\mathrm{P}(w\,\vert \,z)} {\mathrm{P}(w)}$$

(1)

P(w) is called the prior probability, P(w | z) is called the posterior probability, and P(z | w) is called the likelihood.

Discussion

Bayes rule is used for two purposes. The first is Bayesian update. In this context, z represents some new information that has become available since an estimate P(w) was formed of some hypothesis w. The application of Bayes’ rule enables a new estimate of the probability of w (the posterior probability) to be calculated from estimates of the prior probability, the likelihood and P(z).

The second common application of Bayes’ rule is for estimating posterior probabilities in probabilistic learning, where it is the core of Bayesian networks, naïve Bayes, and semi-na...