Shay Hummel
David Carmel
ABSTRACT
We present a novel interpretation of Clarity [5], a widely used query performance predictor. While Clarity is commonly described as a measure of the "distance" between the language model of the top-retrieved documents and that of the collection, we show that it actually quantifies an additional property of the result list, namely, its diversity. This analysis, along with empirical evaluation, helps to explain the low prediction quality of Clarity for large-scale Web collections.
- N. Balasubramanian, G. Kumaran, and V. R. Carvalho. Predicting query performance on the web. In Proceedings of SIGIR, pages 785--786, 2010. Google Scholar
Digital Library
- D. Carmel and E. Yom-Tov. Estimating the Query Difficulty for Information Retrieval. Synthesis lectures on information concepts, retrieval, and services. Morgan & Claypool, 2010. Google ScholarDigital Library
- C. L. A. Clarke, N. Craswell, and I. Soboroff. Overview of the TREC 2009 Web track. In Proceedings of TREC, 2009.Google Scholar
- G. V. Cormack, M. D. Smucker, and C. L. A. Clarke. Efficient and effective spam filtering and re-ranking for large web datasets. CoRR, abs/1004.5168, 2010. Google ScholarDigital Library
- S. Cronen-Townsend, Y. Zhou, and W. B. Croft. Predicting query performance. In Proceedings of SIGIR, pages 299--306, 2002. Google ScholarDigital Library
- C. Hauff, V. Murdock, and R. Baeza-Yates. Improved query difficulty prediction for the web. In Proceedings of CIKM, pages 439--448, 2008. Google ScholarDigital Library
- B. He and I. Ounis. Inferring query performance using pre-retrieval predictors. In Proceedings of SPIRE, pages 43--54, 2004.Google ScholarCross Ref
Index Terms
- Clarity re-visited
Recommendations
Estimating query representativeness for query-performance prediction
SIGIR '13: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrievalThe query-performance prediction (QPP) task is estimating retrieval effectiveness with no relevance judgments. We present a novel probabilistic framework for QPP that gives rise to an important aspect that was not addressed in previous work; namely, the ...
Query-performance prediction: setting the expectations straight
SIGIR '14: Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrievalThe query-performance prediction task has been described as estimating retrieval effectiveness in the absence of relevance judgments. The expectations throughout the years were that improved prediction techniques would translate to improved retrieval ...
Query-Performance Prediction Using Minimal Relevance Feedback
ICTIR '13: Proceedings of the 2013 Conference on the Theory of Information RetrievalThere has been much work on devising query-performance prediction approaches that estimate search effectiveness without relevance judgments (i.e., zero feedback). Specifically, post-retrieval predictors analyze the result list of top-retrieved ...
Comments