Generative and Discriminative Learning

Definition

Generative learning refers alternatively to any classification learning process that classifies by using an estimate of the joint probability P(y,x ) or to any classification learning process that classifies by using estimates of the prior probability P(y) and the conditional probability P(x | y) (Jaakkola and Haussler 1999; Jaakkola et al. 1999; Ng and Jordan 2002; Lasserre et al. 2006; Bishop 2007), where y is a class and x is a description of an object to be classified. Given such models or estimates, it is possible to generate synthetic objects from the joint distribution. Generative learning contrasts to discriminative learning in which a model or estimate of P(y | x) is formed without reference to an explicit estimate of any of $\mathrm{P}(y,\mathbf{x}),\mathrm{P}(\mathbf{x})$, or P(x | y).

It is also common to categorize as discriminative approaches based on a decision function that directly map from input x onto the output y (such as support vector machines, neural...