Abstract
We discuss the idea of modelling the statistical distributions of scores of documents, classified as relevant or non-relevant. Various specific combinations of standard statistical distributions have been used for this purpose. Some theoretical considerations indicate problems with some of the choices of pairs of distributions. Specifically, we revisit a generalisation of the well-known inverse relationship between recall and precision: some choices of pairs of distributions violate this generalised relationship. We identify the choices and the violations, and explore some of the consequences of this theoretical view.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Swets, J.A.: Information retrieval systems. Science 141(3577), 245–250 (1963)
Baumgarten, C.: A probabilistic solution to collection fusion problem in distributed information retrieval. In: Hearst, M., Gey, F., Tong, R. (eds.) SIGIR’99: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 246–253. ACM Press, New York (1999)
Arampatzis, A., van Hameren, A.: The score-distributional threshold optimization for adaptive binary classification tasks. In: Croft, W.B., et al. (eds.) SIGIR 2001: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 285–293. ACM Press, New York (2001)
Manmatha, R., Rath, T., Feng, F.: Modelling score distributions for combining the outputs of search engines. In: Croft, W.B., et al. (eds.) SIGIR 2001: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 267–275. ACM Press, New York (2001)
Swets, J.A.: Effectiveness of information retrieval methods. American Documentation 20, 72–89 (1969)
Bookstein, A.: When the most ‘pertinent’ document should not be retrieved – an analysis of the Swets model. Information Processing and Management 13, 377–383 (1977)
Collins-Thompson, K., et al.: Information filtering, novelty detection and named page finding. In: Voorhees, E.M., Harman, D.K. (eds.) The Eleventh Text REtrieval Conference, TREC 2002. NIST Special Publication, pp. 107–118. NIST (2003)
Robertson, S.E.: Explicit and implicit variables in information retrieval systems. Journal of the American Society for Information Science 26(4), 214–222 (1975)
van Rijsbergen, C.J.: Retrieval effectiveness. In: Voigt, M.J., Hanneman, G.J. (eds.) Progress in communication sciences, vol. 1, pp. 91–118. Ablex Publishing, Norwood (1979)
Hawking, D., Robertson, S.: On collection size and retrieval effectiveness. Information Retrieval 6, 99–150 (2003)
Robertson, S.E.: The probability ranking principle in information retrieval. Journal of Documentation 33, 294–304 (1977)
Robertson, S.E.: The parametric description of retrieval tests. part 1: The basic parameters. Journal of Documentation 25(1), 1–27 (1969)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Robertson, S. (2007). On Score Distributions and Relevance. In: Amati, G., Carpineto, C., Romano, G. (eds) Advances in Information Retrieval. ECIR 2007. Lecture Notes in Computer Science, vol 4425. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71496-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-540-71496-5_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-71494-1
Online ISBN: 978-3-540-71496-5
eBook Packages: Computer ScienceComputer Science (R0)