Negative Correlation Learning

Synonyms

NC-learning; NCL

Definition

Negative correlation learning (Liu and Yao 1999) is an ensemble learning technique. It can be used for regression or classification problems, though with classification problems the models must be capable of producing posterior probabilities. The model outputs are combined with a uniformly weighted average. The squared error is augmented with a penalty term which takes into account the diversity of the ensemble. The error for the ith model is,

$$\displaystyle\begin{array}{rcl} E(f_{i}(x))\,=\,\frac{1} {2}(f_{i}(x) - d)^{2} -\lambda (f_{ i}(x) -\bar{ f}(x))^{2}.& &{}\end{array}$$

(1)

The coefficient λ determines the balance between optimizing individual accuracy, and optimizing ensemble diversity. With λ = 0, the models are trained independently, with no emphasis on diversity. With λ = 1, the models are tightly coupled, and the ensemble is trained as a single unit. Theoretical studies (Brown et al. 2006) have shown that NC works by directly...