Skip to main content
Springer Nature Link
Log in

Indiscriminateness in Representation Spaces of Terms and Documents

  • Conference paper
  • First Online:
Advances in Information Retrieval (ECIR 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10772))

Included in the following conference series:

Abstract

Examining the properties of representation spaces for documents or words in Information Retrieval (IR) – typically \(\mathbb {R}^n\) with n large – brings precious insights to help the retrieval process. Recently, several authors have studied the real dimensionality of the datasets, called intrinsic dimensionality, in specific parts of these spaces [14]. They have shown that this dimensionality is chiefly tied with the notion of indiscriminateness among neighbors of a query point in the vector space. In this paper, we propose to revisit this notion in the specific case of IR. More precisely, we show how to estimate indiscriminateness from IR similarities in order to use it in representation spaces used for documents and words [7, 18]. We show that indiscriminateness may be used to characterize difficult queries; moreover we show that this notion, applied to word embeddings, can help to choose terms to use for query expansion.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Amati, G., Rijsbergen, C.J.V.: Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Trans. Inf. Syst. 20, 357–389 (2002)

    Article  Google Scholar 

  2. Amsaleg, L., Oussama, C., Furon, T., Girard, S., Houle, M.E., Kawarabayashi, K.I.: Estimating local intrinsic dimensionality. In: 21st Conference on Knowledge Discovery and Data Mining, KDD 2015, Sidney, Australia, August 2015. https://hal.inria.fr/hal-01159217

  3. Bellogín, A., de Vries, A.P.: Understanding similarity metrics in neighbour-based recommender systems. In: Proceedings of the 2013 Conference on the Theory of Information Retrieval, ICTIR 2013, pp. 13:48–13:55. ACM, New York (2013). http://doi.acm.org/10.1145/2499178.2499186

  4. Beygelzimer, A., Kakade, S., Langford, J.: Cover trees for nearest neighbors. In: Proceedings of International Conference on Machine Learning (ICML), pp. 97–104 (2006)

    Google Scholar 

  5. Clauset, A., Shalizi, C.R., Newman, M.E.J.: Power-law distributions in empirical data. SIAM Rev. 51(4), 661–703 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  6. Claveau, V., Kijak, E.: Direct vs. indirect evaluation of distributional thesauri. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 1837–1848. The COLING 2016 Organizing Committee, Osaka, December 2016. http://aclweb.org/anthology/C16-1173

  7. Claveau, V., Kijak, E., Ferret, O.: Improving distributional thesauri by exploring the graph of neighbors. In: International Conference on Computational Linguistics, COLING 2014, Dublin, August 2014. https://hal.archives-ouvertes.fr/hal-01027545

  8. Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. 41, 391 (1990)

    Article  Google Scholar 

  9. Fang, H., Tao, T., Zhai, C.: Diagnostic evaluation of information retrieval models. ACM Trans. Inf. Syst. 29, 7 (2011)

    Article  Google Scholar 

  10. Hersh, W., Buckley, C., Leone, T.J., Hickam, D.: OHSUMED: an interactive retrieval evaluation and new large test collection for research. In: Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 1994, pp. 192–201. Springer-Verlag, New York Inc., New York (1994). http://dl.acm.org/citation.cfm?id=188490.188557

  11. Hoffman, M., Bach, F.R., Blei, D.M.: Online learning for latent dirichlet allocation. In: Lafferty, J., Williams, C., Shawe-Taylor, J., Zemel, R., Culotta, A. (eds.) Advances in Neural Information Processing Systems, vol. 23, pp. 856–864. Curran Associates, Inc. (2010). http://papers.nips.cc/paper/3902-online-learning-for-latent-dirichlet-allocation.pdf

  12. Houle, M.E., Ma, X., Nett, M., Oria, V.: Dimensional testing for multi-step similarity search. In: Proceedings of the 12th IEEE International Conference on Data Mining (ICDM), pp. 299–308 (2012)

    Google Scholar 

  13. Houle, M.E., Nett, M.: Rank cover trees for nearest neighbor search. In: Brisaboa, N., Pedreira, O., Zezula, P. (eds.) SISAP 2013. LNCS, vol. 8199, pp. 16–29. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-41062-8_3

    Chapter  Google Scholar 

  14. Houle, M., Kashima, H., Nett, M.: Generalized expansion dimension. In: Proceedings of the 12th IEEE International Conference on Data Mining Workshops (ICDMW), pp. 587–594 (2012)

    Google Scholar 

  15. Levina, E., Bickel, P.J.: Maximum likelihood estimation of intrinsic dimension. In: Advances in Neural Information Processing Systems (NIPS) (2004)

    Google Scholar 

  16. Lv, Y., Zhai, C.: Lower-bounding term frequency normalization. In: Proceedings of the 20th ACM International Conference on Information and Knowledge Managementm, CIKM 2011, pp. 7–16. ACM, New York (2011). http://doi.acm.org/10.1145/2063576.2063584

  17. Metzler, D., Croft, W.: Combining the language model and inference network approaches to retrieval. Inf. Process. Manag. 40(5), 735–750 (2004). Special Issue on Bayesian Networks and Information Retrieval

    Article  Google Scholar 

  18. Mikolov, T., Yih, W.t., Zweig, G.: Linguistic regularities in continuous space word representations. In: 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2013), Atlanta, Georgia, pp. 746–751 (2013)

    Google Scholar 

  19. Ponte, J.M., Croft, W.B.: A language modeling approach to information retrieval. In: Proceedings of the 21st Annual international ACM SIGIR Conference on Research and Development in information Retrieval (SIGIR 1998), pp. 275–281 (1998)

    Google Scholar 

  20. Robertson, S.E., Walker, S., Hancock-Beaulieu, M.: Okapi at TREC-7: automatic ad hoc, filtering, VLC and interactive track. In: Proceedings of the 7th Text Retrieval Conference, TREC-7, pp. 199–210 (1998)

    Google Scholar 

  21. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)

    Article  Google Scholar 

  22. Scholkopf, B., Smola, A.J., Muller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10(5), 1299–1319 (1998)

    Article  Google Scholar 

  23. Singhal, A.: Modern information retrieval: a brief overview. Bull. IEEE Comput. Soc. Tech. Committee Data Eng. 24, 35–43 (2001)

    Google Scholar 

  24. Strohman, T., Metzler, D., Turtle, H., Croft, W.: Indri: a language-model based search engine for complex queries (extended version). Technical report CIIR (2005)

    Google Scholar 

  25. Turtle, H., Croft, W.: Evaluation of an inference network-based retrieval model. ACM Trans. Inf. Syst. 9(3), 187–222 (1991)

    Article  Google Scholar 

  26. Venna, J., Kaski, S.: Local multidimensional scaling. Neural Netw. 19, 889–899 (2006)

    Article  MATH  Google Scholar 

  27. de Vries, T., Chawla, S., Houle, M.E.: Density-preserving projections for large-scale local anomaly detection. Knowl. Inf. Syst. 32(1), 25–52 (2012)

    Article  Google Scholar 

  28. Zhai, C., Lafferty, J.D.: A study of smoothing methods for language models applied to ad hoc information retrieval. In: Proceedings of the SIGIR Conference, pp. 334–342 (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Univ. Rennes, CNRS, IRISA, Campus de Beaulieu, Rennes, France

    Vincent Claveau

Authors
  1. Vincent Claveau
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vincent Claveau .

Editor information

Editors and Affiliations

  1. Department of Informatics, Systems, and Communication, University of Milano-Bicocca, Milan, Italy

    Gabriella Pasi

  2. LIP6 – UPMC/CNRS, University Pierre et Marie Curie, Paris, France

    Benjamin Piwowarski

  3. University of Glasgow, Glasgow, United Kingdom

    Leif Azzopardi

  4. Technical University of Vienna, Vienna, Austria

    Allan Hanbury

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Claveau, V. (2018). Indiscriminateness in Representation Spaces of Terms and Documents. In: Pasi, G., Piwowarski, B., Azzopardi, L., Hanbury, A. (eds) Advances in Information Retrieval. ECIR 2018. Lecture Notes in Computer Science(), vol 10772. Springer, Cham. https://doi.org/10.1007/978-3-319-76941-7_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-76941-7_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-76940-0

  • Online ISBN: 978-3-319-76941-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics