Occam’s Razor

Synonyms

Ockham’s Razor

Definition

Occam’s Razor is the maxim that “entities are not to be multiplied beyond necessity,” or as it is often interpreted in the modern context “of two hypotheses H and H’, both of which explain E, the simpler is to be preferred” (Good 1977)

Motivation and Background

Most attempts to learn a model from data confront the problem that there will be many models that are consistent with the data. In order to learn a single model, a choice must be made between the available models. The factors taken into account by a learner in choosing between models are called its learning biases (Mitchell 1980). A preference for simple models is a common learning bias and is embodied in many learning techniques including pruning, minimum message length, and minimum description length. Regularization is also sometimes viewed as an application of Occam’s razor.

Occam’s razor is an imperative, rather than a proposition. That is, it is neither true nor false. Rather, it is a call...