skip to main content
10.1145/3077136.3080751acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
short-paper
Public Access

DBpedia-Entity v2: A Test Collection for Entity Search

Published: 07 August 2017 Publication History

Abstract

The DBpedia-entity collection has been used as a standard test collection for entity search in recent years. We develop and release a new version of this test collection, DBpedia-Entity v2, which uses a more recent DBpedia dump and a unified candidate result pool from the same set of retrieval models. Relevance judgments are also collected in a uniform way, using the same group of crowdsourcing workers, following the same assessment guidelines. The result is an up-to-date and consistent test collection.To facilitate further research, we also provide details about the pre-processing and indexing steps, and include baseline results from both classical and recently developed entity search methods.

References

[1]
Krisztian Balog and Robert Neumayer 2012. Hierarchical target type identification for entity-oriented queries Proc. of CIKM'12. 2391--2394.
[2]
Krisztian Balog and Robert Neumayer 2013. A Test Collection for Entity Search in DBpedia. Proc. of SIGIR'13. 737--740.
[3]
Krisztian Balog, Pavel Serdyukov, Arjen De Vries, Paul Thomas, and Thijs Westerveld 2010. Overview of the TREC 2009 Entity Track. In Proc. of TREC'09.
[4]
Roi Blanco, Harry Halpin, Daniel M Herzig, Peter Mika, Jeffrey Pound, and Henry S Thompson. 2011. Entity Search Evaluation over Structured Web Data. Proc. of the 1st International Workshop on Entity-Oriented Search. 65--71.
[5]
Jing Chen, Chenyan Xiong, and Jamie Callan 2016. An Empirical Study of Learning to Rank for Entity Search Proc. of SIGIR'16. 737--740.
[6]
Gianluca Demartini, Tereza Iofciu, and Arjen P De Vries. 2009. Overview of the INEX 2009 Entity Ranking Track. In INEX. 254--264.
[7]
Paolo Ferragina and Ugo Scaiella 2010. TAGME: On-the-fly Annotation of Short Text Fragments (by Wikipedia Entities) Proc. of CIKM'10. 1625--1628.
[8]
Dario Garigliotti, Faegheh Hasibi, and Krisztian Balog. 2017. Target Type Identification for Entity-Bearing Queries Proc. of SIGIR'17.
[9]
Jiafeng Guo, Gu Xu, Xueqi Cheng, and Hang Li. 2009. Named Entity Recognition in Query. In Proc. of SIGIR'09. 267--274.
[10]
Harry Halpin, Daniel M Herzig, Peter Mika, Roi Blanco, Jeffrey Pound, Henry S Thompson, and Duc Thanh Tran 2010. Evaluating Ad-hoc Object Retrieval. In Proc. of the International Workshop on Evaluation of Semantic Technologies.
[11]
Faegheh Hasibi, Krisztian Balog, and Svein Erik Bratsberg. 2016. Exploiting Entity Linking in Queries for Entity Retrieval Proc. of ICTIR'16. 171--180.
[12]
Faegheh Hasibi, Krisztian Balog, and Svein Erik Bratsberg. 2017natexlaba. Dynamic Factual Summaries for Entity Cards. In Proc. of SIGIR'17.
[13]
Faegheh Hasibi, Krisztian Balog, and Svein Erik Bratsberg. 2017natexlabb. Entity Linking in Queries: Efficiency vs. Effectiveness Proc. of ECIR'17. 40--53.
[14]
Jinyoung Kim, Xiaobing Xue, and W Bruce Croft. 2009. A Probabilistic Retrieval Model for Semistructured Data Proc. of ECIR'09. 228--239.
[15]
Vanessa Lopez, Christina Unger, Philipp Cimiano, and Enrico Motta 2013. Evaluating Question Answering over Linked Data. Web Semantics: Science, Services and Agents on the World Wide Web, Vol. 21, 0 (2013), 3--13.
[16]
Chunliang Lu, Wai Lam, and Yi Liao 2015. Entity Retrieval via Entity Factoid Hierarchy. In Proc. of ACL'15. 514--523.
[17]
Edgar Meij, Krisztian Balog, and Daan Odijk 2014. Entity Linking and Retrieval for Semantic Search. Proc of. WSDM'14. 683--684.
[18]
Edgar Meij, Marc Bron, Laura Hollink, Bouke Huurnink, and Maarten de Rijke 2011. Mapping Queries to the Linking Open Data Cloud: A Case Study Using DBpedia. Web Semant., Vol. 9, 4 (Dec. 2011), 418--433.
[19]
Donald Metzler and W Bruce Croft 2005. A Markov Random Field Model for Term Dependencies. Proc. of SIGIR'05. 472--479.
[20]
Fedor Nikolaev, Alexander Kotov, and Nikita Zhiltsov. 2016. Parameterized Fielded Term Dependence Models for Ad-hoc Entity Retrieval from Knowledge Graph Proc. of SIGIR'16. 435--444.
[21]
Paul Ogilvie and Jamie Callan 2003. Combining Document Representations for Known-item Search Proc. of SIGIR'03. 143--150.
[22]
Jay M Ponte and W Bruce Croft 1998. A Language Modeling Approach to Information Retrieval Proc. of SIGIR'98. 275--281.
[23]
Stephen E. Robertson and Hugo Zaragoza 2009. The Probabilistic Relevance Framework: BM25 and Beyond. Found. and Trends in IR Vol. 3, 4 (2009), 333--389.
[24]
Fabian M Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: A Core of Semantic Knowledge. In Proc. of WWW'07. 697--706.
[25]
Qiuyue Wang, Jaap Kamps, Georgina Ramírez Camps, Maarten Marx, Anne Schuth, Martin Theobald, Sairam Gurajada, and Arunav Mishra. 2012. Overview of the INEX 2012 Linked Data Track. In CLEF Online Working Notes.
[26]
Chenyan Xiong, Russell Power, and Jamie Callan. 2017. Explicit Semantic Ranking for Academic Search via Knowledge Graph Embedding Proc. of WWW'17. 1271--1279.
[27]
Nikita Zhiltsov, Alexander Kotov, and Fedor Nikolaev. 2015. Fielded Sequential Dependence Model for Ad-Hoc Entity Retrieval in the Web of Data Proc. of SIGIR'15. 253--262.

Cited By

View all
  • (2025)A Knowledge Graph Embedding Model for Answering Factoid Entity QuestionsACM Transactions on Information Systems10.1145/367800343:2(1-27)Online publication date: 24-Jan-2025
  • (2025)Knowledge graph based entity selection framework for ad-hoc retrievalWeb Semantics: Science, Services and Agents on the World Wide Web10.1016/j.websem.2024.10084884:COnline publication date: 18-Feb-2025
  • (2025)MHGTN: Multi-Interaction Heterogeneous Graph Transformer Network for Knowledge Graph CompletionNeurocomputing10.1016/j.neucom.2025.129368624(129368)Online publication date: Apr-2025
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval
August 2017
1476 pages
ISBN:9781450350228
DOI:10.1145/3077136
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 August 2017

Permissions

Request permissions for this article.

Check for updates

Badges

  • Honorable Mention Short Paper

Author Tags

  1. dbpedia
  2. entity retrieval
  3. entity search
  4. semantic search
  5. test collection

Qualifiers

  • Short-paper

Funding Sources

Conference

SIGIR '17
Sponsor:

Acceptance Rates

SIGIR '17 Paper Acceptance Rate 78 of 362 submissions, 22%;
Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)235
  • Downloads (Last 6 weeks)33
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)A Knowledge Graph Embedding Model for Answering Factoid Entity QuestionsACM Transactions on Information Systems10.1145/367800343:2(1-27)Online publication date: 24-Jan-2025
  • (2025)Knowledge graph based entity selection framework for ad-hoc retrievalWeb Semantics: Science, Services and Agents on the World Wide Web10.1016/j.websem.2024.10084884:COnline publication date: 18-Feb-2025
  • (2025)MHGTN: Multi-Interaction Heterogeneous Graph Transformer Network for Knowledge Graph CompletionNeurocomputing10.1016/j.neucom.2025.129368624(129368)Online publication date: Apr-2025
  • (2024)No Query Left Behind: Query Refinement via BacktranslationProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679729(1961-1972)Online publication date: 21-Oct-2024
  • (2024)Veracity Estimation for Entity-Oriented Search with Knowledge GraphsProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679561(1649-1659)Online publication date: 21-Oct-2024
  • (2024)Resources for Combining Teaching and Research in Information Retrieval CourseworkProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657886(1115-1125)Online publication date: 10-Jul-2024
  • (2024)Revisiting Document Expansion and Filtering for Effective First-Stage RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657850(186-196)Online publication date: 10-Jul-2024
  • (2024)I3: Intent-Introspective Retrieval Conditioned on InstructionsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657745(1839-1849)Online publication date: 10-Jul-2024
  • (2024)Benchmark and Neural Architecture for Conversational Entity Retrieval from a Knowledge GraphProceedings of the ACM Web Conference 202410.1145/3589334.3645676(1519-1528)Online publication date: 13-May-2024
  • (2024)A Fast Hop-Biased Approximation Algorithm for the Quadratic Group Steiner Tree ProblemProceedings of the ACM Web Conference 202410.1145/3589334.3645325(312-321)Online publication date: 13-May-2024
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media