D-Measure

Definition

D-measure is a family of evaluation measures for diversified search, where we have a set of known intents (a.k.a. subtopics) {i} for a given topic q and the intent probabilities Pr(i|q), such that ∑_iPr(i|q) = 1. It is assumed that per-intent graded relevance assessments are also available for the topic and that the gain value for each document of relevance l with respect to a particular intent is given by gain_l. For example, let the per-intent gain be 3 for a highly relevant document, 2 for a relevant document, and 1 for a partially relevant document. Given a ranked list, let g_i(r) = gain_l if the document at rank r is l-relevant to intent i, and g_i(r) = 0 otherwise. The global gain at r is defined as GG(r) =∑_iPr(i|q)g_i(r). An ideal list for diversified search is defined by sorting all known relevant documents by the global gain; the global gain at rank r in the ideal list is denoted by GG^∗(r). Then, any graded relevance evaluation measure computed based on GG(r) and GG^∗(r)...