Abstract
DLH is a parameter-free divergence from randomness (DFR) model that is normally deployed as a standalone weighting model for retrieval applications. It assumes a hyper-geometric term frequency (tf) distribution which is reduced to a binomial distribution based on non-uniform term prior distribution. In this paper, we revisit the hyper-geometric model by showing that DLH is equivalent to deriving a Poisson-based DFR model based on a binomial distribution with non-uniform document priors. Moreover, instead of treating DLH as a standalone model, we suggest that the effectiveness of DLH can be improved by adding an idf component, since DLH considers only the tf information for the relevance weighting. Experimental results on standard TREC collections with various search tasks show that the newly proposed model with an additional idf component, called PF1, has comparable retrieval performance with the state-of-the-art probabilistic models, and outperforms them when query expansion is applied.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Amati, G.: Frequentist and bayesian approach to information retrieval. In: Lalmas, M., MacFarlane, A., Rüger, S.M., Tombros, A., Tsikrika, T., Yavlinsky, A. (eds.) ECIR 2006. LNCS, vol. 3936, pp. 13–24. Springer, Heidelberg (2006)
Amati, G., van Rijsbergen, C.J.: Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Trans. Inf. Syst. 20(4), 357–389 (2002)
Craswell, N., Hawking, D.: Overview of the TREC-2003 Web track. In: Proceedings of the Thirteenth Text REtrieval Conference (TREC 2004), Gaithersburg, MD (2004)
Hui, K., He, B., Luo, T., Wang, B.: A comparative study of pseudo relevance feedback for ad-hoc retrieval. In: Amati, G., Crestani, F. (eds.) ICTIR 2011. LNCS, vol. 6931, pp. 318–322. Springer, Heidelberg (2011)
Lavrenko, V., Croft, W.B.: Relevance-based language models. In: SIGIR, pp. 120–127 (2001)
Lv, Y., Zhai, C.: A comparative study of methods for estimating query language models with pseudo feedback. In: CIKM, pp. 1895–1898 (2009)
Ounis, I., Amati, G., Plachouras, V., He, B., Macdonald, C., Johnson, D.: Terrier information retrieval platform. In: Losada, D.E., Fernández-Luna, J.M. (eds.) ECIR 2005. LNCS, vol. 3408, pp. 517–519. Springer, Heidelberg (2005)
Renyi, A.: Foundations of probability. Holden-Day San Francisco (1970)
Robertson, S.E., Sparck-Jones, K.: Relevance weighting of search terms. Journal of the American Society for Information Science 27, 129–146 (1976)
Robertson, S.E., Walker, S.: Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. In: SIGIR, pp. 232–241 (1994)
Robertson, S.E., Walker, S., Hancock-Beaulieu, M., Gatford, M., Payne, A.: Okapi at TREC-4. In: TREC (1995)
Tsagkias, M., de Rijke, M., Weerkamp, W.: Hypergeometric language models for republished article finding. In: SIGIR, pp. 485–494 (2011)
Voorhees, E.: TREC: Experiment and Evaluation in Information Retrieval. The MIT Press (2005)
Ye, Z., Huang, X., He, B., Lin, H.: York University at TREC 2009: Relevance feedback track. In: TREC, Gaithersburg, MD (2009)
Zhai, C.: Statistical language models for information retrieval: A critical review, 137–213 (2008)
Zhai, C., Lafferty, J.D., Lafferty, J.D.: A study of smoothing methods for language models applied to ad hoc information retrieval. In: SIGIR, pp. 334–342 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lu, S., He, B., Xu, J. (2013). Hyper-geometric Model for Information Retrieval Revisited. In: Banchs, R.E., Silvestri, F., Liu, TY., Zhang, M., Gao, S., Lang, J. (eds) Information Retrieval Technology. AIRS 2013. Lecture Notes in Computer Science, vol 8281. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-45068-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-45068-6_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-45067-9
Online ISBN: 978-3-642-45068-6
eBook Packages: Computer ScienceComputer Science (R0)