SUBJECTIVE PROBABILITY AND INFORMATION RETRIEVAL: A REVIEW OF THE PSYCHOLOGICAL LITERATURE

The psychological literature on subjective probability estimation is reviewed to determine the feasibility of designing probabilistic information retrieval systems using such estimates. Their use has been considered by some writers, but psychological issues have not been addressed. Research pertinent to probabilistic information retrieval is examined and implications for probabilistic information retrieval are drawn. It is shown that accurate human probability estimation is possible, both in the laboratory and in real world tasks, e.g., in meteorological forecasting; but that it is also a task subject to systematic bias, or inaccuracy. Proposed techniques for debiasing are considered. The highly task‐dependent nature of such estimates is also discussed; two implications are that results from laboratory studies may have limited relevance to real world tasks and that empirical studies specific to the context of information retrieval need to be made. Human probability estimation appears to be a difficult task, but one which can be done well with proper training and use of debiasing techniques. It is premature to say how useful such estimates would be in probabilistic information retrieval, but their use should not yet be ruled out.