skip to main content
10.1145/3132847.3133085acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
short-paper

An Improved Test Collection and Baselines for Bibliographic Citation Recommendation

Published: 06 November 2017 Publication History

Abstract

The problem of recommending bibliographic citations to an author who is writing an article has been well-studied. However, different researchers have used different datasets to evaluate proposed techniques, and have sometimes reported contradictory findings regarding the relative effectiveness of various approaches. In addition, these datasets are problematic in one way or another (e.g., in terms of size or availability), precluding the possibility of adopting one (or some) of them as standard benchmarks. A recently created test collection that makes use of data from CiteSeerx is large, heterogenous, and publicly available, but has certain other limitations. In this paper, we propose a way to modify this test collection to address these limitations. We also use the improved test collection to establish a set of baseline results using elementary content-based techniques, as well as reference directed indexing.

References

[1]
Joeran Beel, Bela Gipp, and Corinna Breitinger. 2015. Research paper recommender systems -- a literature survey. International Journal on Digital Libraries (2015). http://docear.org/papers/research_paper_recommender_systems_--_a_literature_survey.pdf
[2]
Cornelia Caragea, Adrian Silvescu, Prasenjit Mitra, and C. Lee Giles. 2013. Can't see the forest for the trees?: a citation recommendation system Proc. 13th ACM/IEEE-CS JCDL. ACM, 111--114. http://dl.acm.org/citation.cfm?id=2467743
[3]
Cornelia Caragea, Jian Wu, Alina Ciobanu, Kyle Williams, Juan Fernandez-Ramirez, Hung-Hsuan Chen, Zhaohui Wu, and Lee Giles. 2014. CiteSeer x: A Scholarly Big Dataset. Advances in Information Retrieval. Springer, 311--322.
[4]
C. Lee Giles, Kurt D. Bollacker, and Steve Lawrence. 1998. CiteSeer: An automatic citation indexing system Proc. 3rd ACM conf. Digital Libraries. ACM, 89--98.
[5]
Donna Harman. 2011. Information Retrieval Evaluation. Morgan & Claypool Publishers.
[6]
Wenyi Huang, Saurabh Kataria, Cornelia Caragea, Prasenjit Mitra, C. Lee Giles, and Lior Rokach. 2012. Recommending citations: translating papers into references Proc. CIKM. ACM, 1910--1914. http://dl.acm.org/citation.cfm?id=2398542
[7]
Avishay Livne, Vivek Gokuladas, Jaime Teevan, Susan T. Dumais, and Eytan Adar. 2014. CiteSight: supporting contextual citation recommendation using differential search Proc. SIGIR. ACM Press, 807--816.
[8]
Yang Lu, Jing He, Dongdong Shan, and Hongfei Yan. 2011. Recommending citations with translation model. In Proc. CIKM. ACM, 2017--2020.
[9]
Sean M. McNee, Istvan Albert, Dan Cosley, Prateep Gopalkrishnan, Shyong K. Lam, Al Mamunur Rashid, Joseph A. Konstan, and John Riedl On the Recommending of Citations for Research Papers Proc. of 2002 CSCW. 116--125.
[10]
F. Ricci, L. Rokach, B. Shapira, and P.B. Kantor. 2011. Recommender Systems Handbook. Springer US. showLCCN2010937590
[11]
Anna Ritchie, Stephen Robertson, and Simone Teufel. 2008. Comparing citation contexts for information retrieval Proc. CIKM. ACM, 213--222.
[12]
Anna Ritchie, Simone Teufel, and Stephen Robertson. 2006. Creating a test collection for citation-based IR experiments Proc. HLT-NAACL. ACL, 391--398.
[13]
Anna Ritchie, Simone Teufel, and Stephen Robertson. 2008. Using terms from citations for IR: some first results. Advances in Information Retrieval. 211--221.
[14]
Dwaipayan Roy, Kunal Ray, and Mandar Mitra. 2016. From a Scholarly Big Dataset to a Test Collection for Bibliographic Citation Recommendation Scholarly Big Data: AI Perspectives, Challenges, and Ideas - AAAI Workshop, 2016.
[15]
Trevor Strohman, W. Bruce Croft, and David Jensen. 2007. Recommending citations for academic papers. In Proc. SIGIR. ACM, 705--706.
[16]
Kazunari Sugiyama and Min-Yen Kan. 2010. Scholarly Paper Recommendation via User's Recent Research Interests Proc. 10th JCDL. 29--38.
[17]
Kazunari Sugiyama and Min-Yen Kan. 2013. Exploiting Potential Citation Papers in Scholarly Paper Recommendation Proc. 13th JCDL. 153--162.
[18]
Kazunari Sugiyama and Min-Yen Kan. 2015. Scholarly Paper Recommendation Datasets (NUS). (2015). http://www.comp.nus.edu.sg/ sugiyama/SchPaperRecData.html

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management
November 2017
2604 pages
ISBN:9781450349185
DOI:10.1145/3132847
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2017

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. bibliographic citations
  2. recommender systems
  3. test collections

Qualifiers

  • Short-paper

Conference

CIKM '17
Sponsor:

Acceptance Rates

CIKM '17 Paper Acceptance Rate 171 of 855 submissions, 20%;
Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)4
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)An anatomization of research paper recommender systemEngineering Applications of Artificial Intelligence10.1016/j.engappai.2022.105641118:COnline publication date: 1-Feb-2023
  • (2021)An overview and evaluation of citation recommendation modelsScientometrics10.1007/s11192-021-03909-y126:5(4083-4119)Online publication date: 2-Mar-2021
  • (2020)Paper recommendation based on heterogeneous network embeddingKnowledge-Based Systems10.1016/j.knosys.2020.106438(106438)Online publication date: Sep-2020
  • (2020)A review of citation recommendation: from textual content to enriched contextScientometrics10.1007/s11192-019-03336-0Online publication date: 3-Jan-2020
  • (2020)A graph-based taxonomy of citation recommendation modelsArtificial Intelligence Review10.1007/s10462-020-09819-4Online publication date: 21-Feb-2020
  • (2018)A LSTM based Model for Personalized Context- Aware Citation RecommendationIEEE Access10.1109/ACCESS.2018.2872730(1-1)Online publication date: 2018

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media