Abstract
In information retrieval systems and digital libraries, result presentation is a very important aspect. In this paper, we demonstrate that only a ranked list of documents, thought commonly used by many retrieval systems and digital libraries, is not the best way of presenting retrieval results. We believe, in many situations, an estimated relevance probability score or an estimated relevance score should be provided for every retrieved document by the information retrieval system/digital library. With such information, the usability of the retrieval result can be improved, and the Euclidean distance can be used as a very good system-oriented measure for the effectiveness of retrieval results. The relationship between the Euclidean distance and some ranking-based measures are also investigated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barry, C.L.: User-defined relevance criteria: an exploratory study. Journal of the American Society for Information Science 45(3), 149–159 (1994)
Buckley, C., Voorhees, E.M.: Retrieval evaluation with incomplete information. In: Proceedings of ACM SIGIR Conference, Sheffield, United Kingdom, July 2004, pp. 25–32 (2004)
Calvé, A.L., Savoy, J.: Database merging strategy based on logistic regression. Information Processing & Management 36(3), 341–359 (2000)
Crestnai, F., Wu, S.: Testing the cluster hypothesis in distributed information retrieval. Information Processing & Management 42(5), 1137–1150 (2006)
Harter, S.P.: Variations in relevance assessments and the measure of retrieval effectiveness. Journal of the American Society for Information Science 47(1), 37–49 (1996)
Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems 20(4), 442–446 (2002)
Sparck Jones, K., van Rijisbergen, C.: Report on the need for and provision of an “ideal” information retrieval test collection. Technical report, British library research and development report 5266, Computer laboratory, University of Cambridge, Cambridge, UK (1975)
Lee, J.H.: Analysis of multiple evidence combination. In: Proceedings of the 20th Annual International ACM SIGIR Conference, Philadelphia, Pennsylvania, USA, July 1997, pp. 267–275 (1997)
Manmatha, R., Rath, T., Feng, F.: Modelling score distributions for combining the outputs of search engines. In: Proceedings of the 24th Annual International ACM SIGIR Conference, New Orleans, USA, September 2001, pp. 267–275 (2001)
Montague, M., Aslam, J.A.: Relevance score normalization for metasearch. In: Proceedings of ACM CIKM Conference, Berkeley, USA, November 2001, pp. 427–433 (2001)
Nottelmann, H., Fuhr, N.: From retrieval status values to probabilities of relevance for advanced ir applications. Information Retrieval 6(3-4), 363–388 (2003)
Sanderson, M., Zobel, J.: Information retrieval system evaluation: Effort, sensitivity, and reliability. In: Proceedings of ACM SIGIR Conference, Salvador, Brazil, August 2005, pp. 162–169 (2005)
Saracevic, T.: Relevance: A review of and a framework for thinking on the notion in information science. Journal of the American Society for Information Science 26(6), 321–343 (1975)
Schamber, L., Eisenberg, M.B., Nilan, M.S.: A re-examination of relevance: toward a dynamic, situational definition. Information Processing & Management 26(6), 755–776 (1990)
Tombros, A., Villa, R., van Rijsbergen, C.J.: The effectiveness of query-specific hierarchic clustering in information retrieval. Information Processing & Management 38(4), 559–582 (2002)
TREC, http://trec.nist.gov/
van Rijsbergen, C.J.: Information Retrieval. Butterworths (1979)
Wu, S., Bi, Y., McClean, S.: Regression relevance models for data fusion. In: Proceedings of the 18th International Workshop on Database and Expert Systems Applications, Regensburg, Germany, September 2007, pp. 264–268 (2007)
Wu, S., McClean, S.: Evaluation of system measures for incomplete relevance judgment in IR. In: Larsen, H.L., Pasi, G., Ortiz-Arroyo, D., Andreasen, T., Christiansen, H. (eds.) FQAS 2006. LNCS (LNAI), vol. 4027, pp. 245–256. Springer, Heidelberg (2006)
Zobel, J.: How reliable are the results of large-scale information retrieval experiments. In: Proceedings of ACM SIGIR Conference, Melbourne, Australia, August 1998, pp. 307–314 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wu, S., Bi, Y., Zeng, X. (2010). Retrieval Result Presentation and Evaluation. In: Bi, Y., Williams, MA. (eds) Knowledge Science, Engineering and Management. KSEM 2010. Lecture Notes in Computer Science(), vol 6291. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15280-1_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-15280-1_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15279-5
Online ISBN: 978-3-642-15280-1
eBook Packages: Computer ScienceComputer Science (R0)