A Bayesian approach to user stopping rules for information retrieval systems

doi:10.1016/0306-4573(81)90049-2

Information Processing & Management

Volume 17, Issue 6, 1981, Pages 349-361

https://doi.org/10.1016/0306-4573(81)90049-2 Get rights and content

Abstract

A model of a user's scan of the output of an information storage and retrieval system in response to a query is presented. Rules for determining the user's optimal stopping point are discussed and compared. A dynamic model for determining the proper stopping point using decision theory under risk with changing utilities is used as the basis for a Bayesian model of user scanning behavior. An algorithm to implement the Bayesian model is introduced and examples of the model are given. The implications for retrieval systems design and evaluation are discussed.

References (15)

J.A. Swets
Information retrieval system
Science
(1963)
D.H. Kraft et al.
Evaluation of information retrieval systems: a decision theory approach
J. Am. Soc. Information Sci.
(1978)
W.S. Cooper
Expected search length: a single measure of retrieval effectiveness based on the weak ordering action of retrieval systems
Am. Docum.
(1968)
W.S. Cooper
On selecting a measure of retrieval effectiveness, Part I the subjective philosophy of evaluation, Part II implementation of the philosophy
J. Am. Soc. Information Sci.
(1973)
W.S. Cooper
On selecting a measure of retrieval effectiveness, Part I the subjective philosophy of evaluation, Part II implementation of the philosophy
J. Am. Soc. Information Sci.
(1973)
I. Travis
Design equations: vehicles for research into factors affecting bibliographic reference retrieval system performance
(1975)
D.H. Kraft et al.
Stopping rules and their effect on expected search length
Inform. Proc. & Management
(1979)
J.A. Swets
Effectiveness of information retrieval methods
Am. Docum.
(1969)

There are more references available in the full text version of this article.

Cited by (9)

Estimating effective display size in online retrieval systems
1988, Information Processing and Management
This article outlines a problem in commercial online retrieval systems, provides a review of the relevant literature, and presents a solution for a special case of the problem. Previous investigators have considered how to best determine, for a ranked list of records retrieved from an online retrieval system, whether or not the user should continue to display the output. This article examines the problem of how effective display size can be estimated as a means of assisting the users of commercial online retrieval systems. Although no experimental results are as yet available, the approach presented here will provide a guide to and prolegomenon for systematic study of the problem, as well as a method for providing the estimated number of relevant records remaining in a retrieved set ranked by a retrieval status value.
Predicting document retrieval system performance: an expected precision measure
1987, Information Processing and Management
Document retrieval systems based on probabilistic or fuzzy logic considerations may order documents for retrieval. Users then examine the ordered documents until deciding to stop, based on the estimate that the highest ranked unretrieved document will be most economically not retrieved. We propose an expected precision measure useful in estimating the performance expected if yet unretrieved documents were to be retrieved, providing information that may result in more economical stopping decisions. An expected precision graph, comparing expected precision versus document rank, may graphically display the relative expected precision of retrieved and unretrieved documents and may be used as a stopping aid for online searching of text data bases. The effectiveness of relevance feedback may be examined as a search progresses. Expected precision values may also be used as a cutoff for systems consistent with probabilistic models operating in batch modes. Techniques are given for computing the best expected precision obtainable and the expected precision of subject neutral documents.
Advances in Information Retrieval: Where Is That /#&@¢ Record?
1985, Advances in Computers
This chapter discusses information retrieval system. Much effort goes into determining where to store information and how to retrieve it efficiently and effectively. The information retrieval system is a special class of information systems along with database management and question-answering systems. The key element that distinguishes information retrieval systems is the concept that the relevance of any given record to any specific query cannot be determined exactly. In recent years, many commercialized systems and theoretical approaches have emerged. Nevertheless, information retrieval is an interesting and challenging area of study and application, and it is an area currently in great flux. The chapter begins with discussion of various types of information systems that relate to information retrieval. This will be followed by a general discussion of some systems currently available in the marketplace. Then, the work being done in the areas of content analysis, including natural language applications, query processing, and systems evaluation, will be presented. Finally, current research efforts in related areas, such as artificial intelligence, and fuzzy subset theory, will be presented.
Judgements during information seeking: A naturalistic approach to understanding the assessment of enough information
2008, Journal of Information Science
What defines 'enough' information? How policy workers make judgements and decisions during information seeking: Preliminary results from an exploratory study
2006, Information Research
Are two document clusters better than one? The cluster performance question for information retrieval
2005, Journal of the American Society for Information Science and Technology

View all citing articles on Scopus

View full text

A Bayesian approach to user stopping rules for information retrieval systems

Abstract

Science

Evaluation of information retrieval systems: a decision theory approach

J. Am. Soc. Information Sci.

Expected search length: a single measure of retrieval effectiveness based on the weak ordering action of retrieval systems

Am. Docum.

On selecting a measure of retrieval effectiveness, Part I the subjective philosophy of evaluation, Part II implementation of the philosophy

J. Am. Soc. Information Sci.

On selecting a measure of retrieval effectiveness, Part I the subjective philosophy of evaluation, Part II implementation of the philosophy

J. Am. Soc. Information Sci.

Design equations: vehicles for research into factors affecting bibliographic reference retrieval system performance

Stopping rules and their effect on expected search length

Inform. Proc. & Management

Effectiveness of information retrieval methods

Am. Docum.