Inverse Document Frequency

Synonyms

IDF

Definition

The inverse document frequency (IDF) is a statistical weight used for measuring the importance of a term in a text document collection. The document frequency DF of a term is defined by the number of documents in which a term appears.

Key Points

Karen Sparck-Jones first proposed that terms with low document frequency are more valuable than terms with high document frequency during retrieval [2]. In other words, the underlying idea of IDF is that the more frequently the term appears in the collection, the less informative the term is.

In its simplest form, the IDF weight of a term is assigned as follows [3]:

$$IDF = \log_2 {N \over DF}$$

((1))

where N is the number of documents in the collection, and DF is the document frequency of the term, i.e., the number of documents in which the term appears.

There have been different variations of the IDF weight in the literature. For example, Robertson and Walker proposed the following formula [1]:

$$IDF = \log {N - DF +...