Skip to main content
SpringerLink
Log in

A review of citation recommendation: from textual content to enriched context

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Citation recommendation systems play an important role to alleviate the dilemma that scholar users spend a lot of time and experiences for literature survey. With the burgeoning computational models and open data movement, scientific repository can provide more evidence in support of recommendation. On the one hand, recommenders are applying better algorithms to understand the text of user queries and candidate citations. On the other hand, more types of data such as citation network and co-author relationship are aggregated to enrich the citation contextual information. The available data used for recommendation has been extended from textual content to enriched context. This review is conducted to identify the information and methods used for recommendations recently. We begin by introducing definitions of the task, recommending factors along with the corresponding problems and some application platforms. Then, we classify existing recommenders according to user query types and review representative methods for each type. We also elaborate on different strategies applied in three main stages of citation recommendation. Finally, a few open issues for future investigations are proposed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Available at: https://scholar.google.com/.

  2. Available at: http://citeseerx.ist.psu.edu/index.

  3. Available at: https://www.ncbi.nlm.nih.gov/pubmed/.

  4. Available at: https://www.researchgate.net/.

  5. Available at: https://www.mendeley.com/.

  6. Available at: http://www.citeulike.org/.

  7. Available at: http://www.docear.org.

  8. Available at: https://link.springer.com/.

  9. Available at: https://www.sciencedirect.com/.

  10. Available at: https://dl.acm.org/.

  11. Available at: https://scholar.google.com/.

  12. Database retrieval time is from 5th August 2018 to 6th August 2018. We also added few relevant papers using query of paper recommendation during retrieval procedure.

  13. Available at: http://theadvisor.osu.edu.

  14. Available at: http://www.docear.org.

  15. http://www.citeulike.org/faq/data.adp.

  16. https://data.mendeley.com/.

  17. https://dblp.uni-trier.de/.

  18. https://catalog.data.gov/dataset/pubmed.

  19. https://dbs.uni-leipzig.de/de/research/projects/object_matching/fever/benchmark_datasets_for_entity_resolution.

  20. https://aminer.org/open-academic-graph.

References

  • Abbes, R., Pinel-Sauvagnat, K., Hernandez, N., & Boughanem, M. (2013). Irit at trec knowledge base acceleration 2013: Cumulative citation recommendation task. In The twenty-second text retrieval conference-TREC 2013 (pp. 1–4).

  • Alotaibi, S., & Vassileva, J. (2015). Multi-dimensional ratings for research paper recommender systems: A qualitative study. In International symposium on web algorithms.

  • Amami, M. (2018). Topic modeling for scientific paper recommendation. Ph.D. Dissertation. University of Milan-Bicocca, Italy.

  • Amami, M., Faiz, R., Stella, F., & Pasi, G. (2017). A graph based approach to scientific paper recommendation. In Proceedings of the international conference on web intelligence (pp. 777–782). ACM.

  • Amami, M., Pasi, G., Stella, F., & Faiz, R. (2016). An lda-based approach to scientific paper recommendation. In International conference on applications of natural language to information systems (pp. 200–210). Springer.

  • Anand, A., Chakraborty, T., & Das, A. (2017). Fairscholar: Balancing relevance and diversity for scientific paper recommendation. In European conference on information retrieval (pp. 753–757). Springer.

  • Arif, M. A. (2016). Content aware citation recommendation system. In International conference on emerging technological trends (ICETT) (pp. 1–6). IEEE.

  • Balog, K., & Ramampiaro, H. (2013). Cumulative citation recommendation: Classification vs. ranking. In Proceedings of the 36th international ACM SIGIR conference on research and development in information retrieval (pp. 941–944). ACM.

  • Balog, K., Ramampiaro, H., Takhirov, N., & Nørvåg, K. (2013). Multi-step classification approaches to cumulative citation recommendation. In Proceedings of the 10th conference on open research areas in information retrieval (pp. 121–128). LE CENTRE DE HAUTES ETUDES INTERNATIONALES D’INFORMATIQUE DOCUMENTAIRE.

  • Beel, J., Gipp, B., Langer, S., & Breitinger, C. (2016). Paper recommender systems: a literature survey. International Journal on Digital Libraries,17(4), 305–338.

    Article  Google Scholar 

  • Beel, J., & Langer, S. (2015). A comparison of offline evaluations, online evaluations, and user studies in the context of research-paper recommender systems. In International conference on theory and practice of digital libraries (pp. 153–168). Springer.

  • Beel, J., Langer, S., Genzmehr, M., & Nürnberger, A. (2013). Introducing Docear’s research paper recommender system. In JCDL (pp. 459–460).

  • Beel, J., Langer, S., Gipp, B., & Nürnberger, A. (2014a). The architecture and datasets of Docear’s Research paper recommender system. D-Lib Magazine, 20(11/12), 1–13.

    Article  Google Scholar 

  • Beel, J., Langer, S., Kapitsaki, G. M., & GIPP, B. (2014b). Mind-map based user modeling and research paper recommender systems (preprint). 1–27. Retrieved August 2018 from https://www.researchgate.net/profile/Joeran_Beel2/publication/265551384_Mind-Map_based_User_Modelling_and_Research_Paper_Recommendations/links/5411ca3e0cf2fa878ad38e78.pdf

  • Berger, A., & Lafferty, J. (2017). Information retrieval as statistical translation. In ACM SIGIR forum (Vol. 51, no. 2, pp. 219–226). ACM.

  • Bhagavatula, C., Feldman, S., Power, R., & Ammar, W. (2018). Content-based citation recommendation. arXiv preprint arXiv:1802.08301.

  • Bornmann, L., & Mutz, R. (2015). Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references. Journal of the Association for Information Science and Technology,66(11), 2215–2222. https://doi.org/10.1002/asi.23329.

    Article  Google Scholar 

  • Breese, J. S., Heckerman, D., & Kadie, C. (1998). Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of the fourteenth conference on uncertainty in artificial intelligence (pp. 43–52). Morgan Kaufmann Publishers Inc.

  • Burke, R. (2007). Hybrid web recommender systems. In The adaptive web (pp. 377–408). Springer.

  • Cai, X., Han, J., Li, W., Zhang, R., Pan, S., & Yang, L. (2018a). A three-layered mutually reinforced model for personalized citation recommendation. IEEE Transactions on Neural Networks and Learning Systems (99), 1–12.

  • Cai, X., Han, J., & Yang, L. (2018b). Generative adversarial network based heterogeneous bibliographic network representation for personalized citation recommendation. In Thirty-second AAAI conference on artificial intelligence.

  • Caragea, C., Silvescu, A., Mitra, P., & Giles, C. L. (2013). Can’t see the forest for the trees? A citation recommendation system. In Proceedings of the 13th ACM/IEEE-CS joint conference on digital libraries (pp. 111–114). ACM.

  • Castells, P., Vargas, S., & Wang, J. (2011). Novelty and diversity metrics for recommender systems: Choice, discovery and relevance. In: Proceedings of International Workshop on Diversity in Document Retrieval (DDR) (pp. 29–37). New York: ACM Press.

  • Chakrabarti, S., Khanna, R., Sawant, U., & Bhattacharyya, C. (2008). Structured learning for non-smooth ranking losses. In Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 88–96). ACM.

  • Chakraborty, T., Modani, N., Narayanam, R., & Nagar, S. (2015). Discern: A diversified citation recommendation system for scientific queries. In IEEE 31st international conference on data engineering (pp. 555–566). IEEE.

  • Chen, J., & Ban, Z. (2016). Literature recommendation by researchers’ publication analysis. In IEEE international conference on information and automation (ICIA) (pp. 1964–1969). IEEE.

  • Chen, C.-H., Mayanglambam, S. D., Hsu, F.-Y., Lu, C.-Y., Lee, H.-M., & Ho, J.-M. (2011). Novelty paper recommendation using citation authority diffusion. In International conference on technologies and applications of artificial intelligence (pp. 126–131). IEEE.

  • Dai, T., Zhu, L., Cai, X., Pan, S., & Yuan, S. (2018). Explore semantic topics and author communities for citation recommendation in bipartite bibliographic network. Journal of Ambient Intelligence and Humanized Computing,9(4), 957–975.

    Article  Google Scholar 

  • Daim, T. U., Rueda, G., Martin, H., & Gerdsri, P. (2006). Forecasting emerging technologies: Use of bibliometrics and patent analysis. Technological Forecasting and Social Change,73(8), 981–1012.

    Article  Google Scholar 

  • Dietz, L., Dalton, J., & Balog, K. (2013). Time-aware evaluation of cumulative citation recommendation systems. In Proceedings of SIGIR 2013 workshop on time-aware information access, TAIA (Vol. 2013).

  • Dinesh, S. (2018). Real world evaluation of approaches to research paper recommendation. arXiv preprint arXiv:1802.06892.

  • Ebesu, T., & Fang, Y. (2017). Neural citation network for context-aware citation recommendation. In Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval (pp. 1093–1096). ACM.

  • Ekstrand, M. D., Kannan, P., Stemper, J. A., Butler, J. T., Konstan, J. A., & Riedl, J. T. (2010). Automatically building research reading lists. In Proceedings of the fourth ACM conference on recommender systems (pp. 159–166). ACM.

  • El-Arini, K., & Guestrin, C. (2011). Beyond keyword search: Discovering relevant scientific literature. In Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 439–447). ACM.

  • Ferrara, F., Pudota, N., & Tasso, C. (2011). A keyphrase-based paper recommender system. In Italian research conference on digital libraries (pp. 14–25). Springer.

  • Figueroa, C., Vagliano, I., Rocha, O. R., & Morisio, M. (2015). A systematic literature review of linked data-based recommender systems. Concurrency and Computation: Practice and Experience,27(17), 4659–4684.

    Article  Google Scholar 

  • Fu, T., Lei, Z., & Lee, W.-C. (2015). Patent citation recommendation for examiners. In IEEE international conference on data mining (pp. 751–756). IEEE.

  • Gao, Z. (2015). Examining influences of publication dates on citation recommendation systems. In 12th international conference on fuzzy systems and knowledge discovery (FSKD) (pp. 1400–1405). IEEE.

  • Ge, M., Delgado-Battenfeld, C., & Jannach, D. (2010). Beyond accuracy: Evaluating recommender systems by coverage and serendipity. In Proceedings of the fourth ACM conference on recommender systems (pp. 257–260). ACM.

  • Gebremeskel, G. G., He, J., De Vries, A. P., & Lin, J. (2014). Cumulative citation recommendation: A feature-aware comparison of approaches. In 25th International workshop on database and expert systems applications (pp. 193–197). IEEE.

  • Gleason, A. M. (1957). Measures on the closed subspaces of a Hilbert space. Journal of Mathematics and Mechanics,6, 885–893.

    MathSciNet  MATH  Google Scholar 

  • Grimes, C., Tang, D., & Russell, D. (2007). Query logs alone are not enough.

  • Gunawardana, A., & Shani, G. (2009). A survey of accuracy evaluation metrics of recommendation tasks. Journal of Machine Learning Research,10, 2935–2962.

    MathSciNet  MATH  Google Scholar 

  • Guo, L., Cai, X., Hao, F., Mu, D., Fang, C., & Yang, L. (2017). Exploiting fine-grained co-authorship for personalized citation recommendation. IEEE Access,5, 12714–12725.

    Article  Google Scholar 

  • Gupta, S., & Varma, V. (2017). Scientific article recommendation by using distributed representations of text and graph. In Proceedings of the 26th international conference on world wide web companion (pp. 1267–1268). International World Wide Web Conferences Steering Committee.

  • Haruna, K., & Ismail, M. A. (2016). An ontological framework for research paper recommendation. International Journal of Soft Computing,11(2), 96–99.

    Google Scholar 

  • Hassan, H. A. M. (2017). Personalized research paper recommendation using deep learning. In Proceedings of the 25th conference on user modeling, adaptation and personalization (pp. 327–330). ACM.

  • He, Q., Kifer, D., Pei, J., Mitra, P., & Giles, C. L. (2011). Citation recommendation without author supervision. Paper presented at the Proceedings of the fourth ACM international conference on WEB search and data mining, Hong Kong, China.

  • He, J., Nie, J.-Y., Lu, Y., & Zhao, W. X. (2012). Position-aligned translation model for citation recommendation. In International symposium on string processing and information retrieval (pp. 251–263). Springer.

  • He, Q., Pei, J., Kifer, D., Mitra, P., & Giles, L. (2010). Context-aware citation recommendation. Paper presented at the Proceedings of the 19th international conference on World wide web, Raleigh, North Carolina, USA.

  • Hong, K., Jeon, H., & Jeon, C. (2013). Personalized research paper recommendation system using keyword extraction based on userprofile. Journal of Convergence Information Technology,8(16), 106.

    Google Scholar 

  • Huang, W., Kataria, S., Caragea, C., Mitra, P., Giles, C. L., & Rokach, L. (2012). Recommending citations: Translating papers into references. In CIKM (Vol. 12, pp. 1910–1914). Citeseer.

  • Huang, W., Wu, Z., Liang, C., Mitra, P., & Giles, C. L. (2015). A neural probabilistic model for context based citation recommendation. In Twenty-ninth AAAI conference on artificial intelligence.

  • Huang, W., Wu, Z., Mitra, P., & Giles, C. L. (2014). RefSeer: A citation recommendation system. In IEEE/ACM joint conference on digital libraries (pp. 371–374). IEEE.

  • Jiang, Z., Liu, X., & Gao, L. (2014). Dynamic topic/citation influence modeling for chronological citation recommendation. In Proceedings of the 5th international workshop on web-scale knowledge representation retrieval and reasoning (pp. 15–18). ACM.

  • Jiang, Z., Liu, X., & Gao, L. (2015). Chronological citation recommendation with information-need shifting. In Proceedings of the 24th ACM international on conference on information and knowledge management (pp. 1291–1300). ACM.

  • Jiang, Z., Lu, Y., & Liu, X. (2018a). Cross-language citation recommendation via publication content and citation representation fusion. In Proceedings of the 18th ACM/IEEE on joint conference on digital libraries (pp. 347–348). ACM.

  • Jiang, Z., Yin, Y., Gao, L., Lu, Y., & Liu, X. (2018b). Cross-language citation recommendation via hierarchical representation learning on heterogeneous graph. In The 41st international ACM SIGIR conference on research and development in information retrieval (pp. 635–644). ACM.

  • Johnson, R., Watkinson, A., & Mabe, M. (2018). The STM report. An overview of scientific and scholarly publishing (5th ed.).

  • Kazemi, B., & Abhari, A. (2017). A comparative study on content-based paper-to-paper recommendation approaches in scientific literature. In Proceedings of the 20th communications and networking symposium (p. 5). Society for Computer Simulation International.

  • Kim, M. C., & Chen, C. (2015). A scientometric review of emerging trends and new developments in recommendation systems. Scientometrics,104(1), 239–263.

    Article  Google Scholar 

  • Kleinberg, J. M. (1999). Authoritative sources in a hyperlinked environment. Journal of the ACM (JACM),46(5), 604–632.

    Article  MathSciNet  MATH  Google Scholar 

  • Knoth, P., & Zdrahal, Z. (2012). CORE: Three access levels to underpin open access. D-Lib Magazine, 18(11/12), 1–13.

    Article  Google Scholar 

  • Kobayashi, Y., Shimbo, M., & Matsumoto, Y. (2018). Citation recommendation using distributed representation of discourse facets in scientific articles. In Proceedings of the 18th ACM/IEEE on joint conference on digital libraries (pp. 243–251). ACM.

  • Kong, X., Mao, M., Wang, W., Liu, J., & Xu, B. (2018). VOPRec: Vector representation learning of papers with text information and structural identity for recommendation. IEEE Transactions on Emerging Topics in Computing, 6(2), 1–12. https://doi.org/10.1109/TETC.2018.2830698.

    Article  Google Scholar 

  • Konstan, J. A., & Riedl, J. (2012). Recommender systems: From algorithms to user experience. User Modeling and User-Adapted Interaction,22(1–2), 101–123.

    Article  Google Scholar 

  • Küçüktunç, O., Saule, E., & Kaya, K. (2013a). Towards a personalized, scalable, and exploratory academic recommendation service. In IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM 2013) (pp. 636–641). IEEE.

  • Küçüktunç, O., Saule, E., Kaya, K., & Çatalyürek, Ü. V. (2012a). Direction awareness in citation recommendation.

  • Küçüktunç, O., Saule, E., Kaya, K., & Çatalyürek, Ü. V. (2012b). Recommendation on academic networks using direction aware citation analysis. arXiv preprint arXiv:1205.1143.

  • Küçüktunç, O., Saule, E., Kaya, K., & Çatalyürek, Ü. V. (2013b). Result diversification in automatic citation recommendation. In Proceedings of the iConference workshop on computational scientometrics: Theory and applications (pp. 1–4).

  • Küçüktunç, O., Saule, E., Kaya, K., & Çatalyürek, Ü. V. (2013c). TheAdvisor: A webservice for academic recommendation. In Proceedings of the 13th ACM/IEEE-CS joint conference on digital libraries (pp. 433–434). ACM.

  • Küçüktunç, O., Saule, E., Kaya, K., & Çatalyürek, Ü. V. (2015). Diversifying citation recommendations. ACM Transactions on Intelligent Systems and Technology (TIST),5(4), 55.

    Google Scholar 

  • Lee, J., Lee, K., & Kim, J. G. (2013). Personalized academic research paper recommendation system. arXiv preprint arXiv:1304.5457.

  • Lee, J., Lee, K., Kim, J. G., & Kim, S. (2015). Personalized academic paper recommendation system. SRS’15.

  • Liang, Y., Li, Q., & Qian, T. (2011). Finding relevant papers based on citation relations. In International conference on web-age information management (pp. 403–414). Springer.

  • Liu, X.-Y., & Chien, B.-C. (2017). Applying citation network analysis on recommendation of research paper collection. In Proceedings of the 4th multidisciplinary international social networks conference (p. 30). ACM.

  • Liu, H., Kong, X., Bai, X., Wang, W., Bekele, T. M., & Xia, F. (2015). Context-based collaborative filtering for citation recommendation. IEEE Access,3, 1695–1703.

    Article  Google Scholar 

  • Liu, X., Yu, Y., Guo, C., & Sun, Y. (2014a). Meta-path-based ranking with pseudo relevance feedback on heterogeneous graph for citation recommendation. In Proceedings of the 23rd ACM international conference on conference on information and knowledge management (pp. 121–130). ACM.

  • Liu, X., Yu, Y., Guo, C., Sun, Y., & Gao, L. (2014b). Full-text based context-rich heterogeneous network mining approach for citation recommendation. In IEEE/ACM joint conference on digital libraries (pp. 361–370). IEEE.

  • Livne, A., Gokuladas, V., Teevan, J., Dumais, S. T., & Adar, E. (2014). CiteSight: Supporting contextual citation recommendation using differential search. In Proceedings of the 37th international ACM SIGIR conference on research and development in information retrieval (pp. 807–816). ACM.

  • Lu, Y., He, J., Shan, D., & Yan, H. (2011). Recommending citations with translation model. In Proceedings of the 20th ACM international conference on information and knowledge management (pp. 2017–2020). ACM.

  • Ma, K., Lu, T., & Abraham, A. (2014). Hybrid parallel approach for personalized literature recommendation system. In 6th International conference on computational aspects of social networks (pp. 31–36). IEEE.

  • Ma, L., Song, D., Liao, L., & Ni, Y. (2017). A joint deep model of entities and documents for cumulative citation recommendation. Cluster Computing, 22, 5435–5446. https://doi.org/10.1007/s10586-017-1273-x.

    Article  Google Scholar 

  • Magara, M. B., Ojo, S., & Zuva, T. (2017). Toward altmetric-driven research-paper recommender system framework. In 13th International conference on signal-image technology and internet-based systems (SITIS) (pp. 63–68). IEEE.

  • Magara, M. B., Ojo, S. O., & Zuva, T. (2018). A comparative analysis of text similarity measures and algorithms in research paper recommender systems. In Conference on information communications technology and society (ICTAS) (pp. 1–5). IEEE.

  • Mahdabi, P., & Crestani, F. (2014). Query-driven mining of citation networks for patent citation retrieval and recommendation. In Proceedings of the 23rd ACM international conference on conference on information and knowledge management (pp. 1659–1668). ACM.

  • Manouselis, N., & Verbert, K. (2013). Layered evaluation of multi-criteria collaborative filtering for scientific paper recommendation. Procedia Computer Science,18, 1189–1197.

    Article  Google Scholar 

  • McNee, S. M., Albert, I., Cosley, D., Gopalkrishnan, P., Lam, S. K., Rashid, A. M., et al. (2002). On the recommending of citations for research papers. In Proceedings of the 2002 ACM conference on computer supported cooperative work (pp. 116–125). ACM.

  • McNee, S. M., Kapoor, N., & Konstan, J. A. (2006). Don’t look stupid: Avoiding pitfalls when recommending research papers. In Proceedings of the 2006 20th anniversary conference on computer supported cooperative work (pp. 171–180). ACM.

  • Meng, F., Gao, D., Li, W., Sun, X., & Hou, Y. (2013). A unified graph model for personalized query-oriented reference paper recommendation. In Proceedings of the 22nd ACM international conference on information and knowledge management (pp. 1509–1512). ACM.

  • Mönnich, M., & Spiering, M. (2008). Adding value to the library catalog by implementing a recommendation system. D-Lib Magazine,14(5/6), 1082–9873.

    Article  Google Scholar 

  • Mu, D., Guo, L., Cai, X., & Hao, F. (2018). Query-focused personalized citation recommendation with mutually reinforced ranking. IEEE Access,6, 3107–3119.

    Article  Google Scholar 

  • Naak, A., Hage, H., & Aimeur, E. (2009). A multi-criteria collaborative filtering approach for research paper recommendation in papyres. In International conference on e-technologies (pp. 25–39). Springer.

  • Nascimento, C., Laender, A. H., da Silva, A. S., & Gonçalves, M. A. (2011). A source independent framework for research paper recommendation. In Proceedings of the 11th annual international ACM/IEEE joint conference on digital libraries (pp. 297–306). ACM.

  • Oard, D. W., & Kim, J. (1998). Implicit feedback for recommender systems. In Proceedings of the AAAI workshop on recommender systems (Vol. 83): WoUongong.

  • Oh, S., Lei, Z., Lee, W.-C., Mitra, P., & Yen, J. (2013). CV-PCR: A context-guided value-driven framework for patent citation recommendation. In Proceedings of the 22nd ACM international conference on information and knowledge management (pp. 2291–2296). ACM.

  • Ohta, M., Hachiki, T., & Takasu, A. (2011). Related paper recommendation to support online-browsing of research papers. In Fourth international conference on the applications of digital information and web technologies (ICADIWT 2011) (pp. 130–136). IEEE.

  • Ollagnier, A., Fournier, S., & Bellot, P. (2018). BIBLME RecSys: Harnessing bibliometric measures for a scholarly paper recommender system. In BIR 2018 workshop on bibliometric-enhanced information retrieval.

  • Pan, L., Dai, X., Huang, S., & Chen, J. (2015). Academic paper recommendation based on heterogeneous graph. In Chinese computational linguistics and natural language processing based on naturally annotated big data (pp. 381–392). Springer.

  • Pan, C., & Li, W. (2010). Research paper recommendation with topic analysis. In International conference on computer design and applications (Vol. 4, pp. V4-264–V264-268). IEEE.

  • Park, D. H., Kim, H. K., Choi, I. Y., & Kim, J. K. (2012). A literature review and classification of recommender systems research. Expert Systems with Applications,39(11), 10059–10072.

    Article  Google Scholar 

  • Pemantle, R. (1992). Vertex-reinforced random walk. Probability Theory and Related Fields,92(1), 117–136.

    Article  MathSciNet  MATH  Google Scholar 

  • Philip, S., Shola, P., & Ovye, A. (2014). Application of content-based approach in research paper recommendation system for a digital library. International Journal of Advanced Computer Science and Applications, 5(10), 37–40.

    Article  Google Scholar 

  • Pruitikanee, S., Di Jorio, L., Laurent, A., & Sala, M. (2012). Paper recommendation system: A global and soft approach. In Future computing.

  • Raamkumar, A. S., Foo, S., & Pang, N. (2016). A framework for scientific paper retrieval and recommender systems. arXiv preprint arXiv:1609.01415.

  • Raamkumar, A. S., Foo, S., & Pang, N. (2017). Using author-specified keywords in building an initial reading list of research papers in scientific paper retrieval and recommender systems. Information Processing and Management,53(3), 577–594.

    Article  Google Scholar 

  • Ravi, K. M., Mori, J., & Sakata, I. (2017). Cross-domain academic paper recommendation by semantic linkage approach using text analysis and recurrent neural networks. In Portland international conference on management of engineering and technology (PICMET) (pp. 1–10). IEEE.

  • Ray, K. (2015). The bibliographic citation recommendation problem. Kolkata: Indian Statistical Institute.

    Google Scholar 

  • Ren, X. (2016). Effective citation recommendation by information network-based clustering. Master's thesis. University of Illinois at Urbana-Champaign, Urbana, IL, USA.

  • Ren, X., Liu, J., Yu, X., Khandelwal, U., Gu, Q., Wang, L., et al. (2014). Cluscite: Effective citation recommendation by information network-based clustering. In Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 821–830). ACM.

  • Resnick, P., & Varian, H. R. (1997). Recommender systems. Communications of the ACM,40(3), 56–59.

    Article  Google Scholar 

  • Ricci, F., Rokach, L., & Shapira, B. (2011). Introduction to recommender systems handbook. In Recommender systems handbook (pp. 1–35). Springer.

  • Robins, D. (1997). Shifts of focus in information retrieval interaction. In Proceedings of the annual meeting American Society for information science (Vol. 34, pp. 123–134). Learned Information (Europe) LTD.

  • Rosvall, M., & Bergstrom, C. T. (2011). Multilevel compression of random walks on networks reveals hierarchical organization in large integrated systems. PLoS ONE,6(4), e18209.

    Article  Google Scholar 

  • Roy, D. (2017). An improved test collection and baselines for bibliographic citation recommendation. In Proceedings of the 2017 ACM on conference on information and knowledge management (pp. 2271–2274). ACM.

  • Roy, D., Ray, K., & Mitra, M. (2016). From a scholarly big dataset to a test collection for bibliographic citation recommendation. In Workshops at the thirtieth AAAI conference on artificial intelligence.

  • Schafer, J. B., Konstan, J., & Riedl, J. (1999). Recommender systems in e-commerce. In Proceedings of the 1st ACM conference on electronic commerce (pp. 158–166). ACM.

  • Schwartz, T., Berger, M., & Hernandez, J. (2015). A legal citation recommendation engine using topic modeling and semantic similarity. In Law and big data workshop, 15th international conference on artificial intelligence and law.

  • Sesagiri Raamkumar, A., & Foo, S. (2018). Multi-method evaluation in scientific paper recommender systems. In Adjunct Publication of the 26th conference on user modeling, adaptation and personalization (pp. 179–182). ACM.

  • Sesagiri Raamkumar, A., Foo, S., & Pang, N. (2015). Rec4LRW–Scientific paper recommender system for literature review and writing. In Proceedings of the 6th international conference on applications of digital information and web technologies (pp. 106–119).

  • Sesagiri Raamkumar, A., Foo, S., & Pang, N. (2016a). Making literature review and manuscript writing tasks easier for novice researchers through Rec4LRW system. In Proceedings of the 16th ACM/IEEE-CS on joint conference on digital libraries (pp. 229–230). ACM.

  • Sesagiri Raamkumar, A., Foo, S., & Pang, N. (2016b). Proposing a scientific paper retrieval and recommender framework. In International conference on Asia-Pacific digital libraries, ICADL, Tsukuba, Japan.

  • Sharda, D., & Dawgotra, P. (2017). Design of research buddy: Personalized research paper recommendation system. Advance Research in Science and Technology,6(9), 1342–1352.

    Google Scholar 

  • Siebert, S., Dinesh, S., & Feyer, S. (2017). Extending a research-paper recommendation system with scientometric measures. In Proceedings of the 5th Workshop on Bibliometric-enhanced Information Retrieval (BIR), pp. 112–121.

  • Son, J., & Kim, S. B. (2018). Academic paper recommender system using multilevel simultaneous citation networks. Decision Support Systems,105, 24–33.

    Article  Google Scholar 

  • Stock, K., Karasova, V., Robertson, A., Roger, G., Small, M., Bishr, M., et al. (2013). Finding science with science: Evaluating a domain and scientific ontology user interface for the discovery of scientific resources. Transactions in GIS,17(4), 612–639.

    Article  Google Scholar 

  • Stock, K., Robertson, A., Reitsma, F., Stojanovic, T., Bishr, M., Medyckyj-Scott, D., et al. (2009). eScience for sea science: A semantic scientific knowledge infrastructure for marine scientists. In Fifth IEEE international conference on e-science (pp. 110–117). IEEE.

  • Sugiyama, K., & Kan, M.-Y. (2010). Scholarly paper recommendation via user’s recent research interests. In Proceedings of the 10th annual joint conference on digital libraries (pp. 29–38). ACM.

  • Sugiyama, K., & Kan, M.-Y. (2011). Serendipitous recommendation for scholarly papers considering relations among researchers. In Proceedings of the 11th annual international ACM/IEEE joint conference on digital libraries (pp. 307–310). ACM.

  • Sugiyama, K., & Kan, M.-Y. (2013). Exploiting potential citation papers in scholarly paper recommendation. In Proceedings of the 13th ACM/IEEE-CS joint conference on digital libraries (pp. 153–162). ACM.

  • Sun, J., Jiang, Y., Cheng, X., Du, W., Liu, Y., & Ma, J. (2018). A hybrid approach for article recommendation in research social networks. Journal of Information Science,44(5), 696–711.

    Article  Google Scholar 

  • Sun, J., Ma, J., Liu, X., Liu, Z., Wang, G., Jiang, H., et al. (2013). A novel approach for personalized article recommendation in online scientific communities. In 46th Hawaii international conference on system sciences (pp. 1543–1552). IEEE.

  • Sun, Y., Ni, W., & Men, R. (2009). A personalized paper recommendation approach based on web paper mining and reviewer’s interest modeling. In International conference on research challenges in computer science (pp. 49–52). IEEE.

  • Tang, X., Wan, X., & Zhang, X. (2014). Cross-language context-aware citation recommendation in scientific articles. Paper presented at the Proceedings of the 37th international ACM SIGIR conference on research and development in information retrieval, Gold Coast, Queensland, Australia.

  • Tang, J., & Zhang, J. (2009). A discriminative approach to topic-based citation recommendation. In T. Theeramunkong, B. Kijsirikul, N. Cercone, & T.-B. Ho (Eds.), Advances in knowledge discovery and data mining: 13th Pacific-Asia conference, PAKDD 2009 Bangkok, Thailand, April 27–30, 2009 Proceedings (pp. 572–579). Berlin: Springer.

  • Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., & Su, Z. (2008). Arnetminer: Extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 990–998). ACM.

  • Tantanasiriwong, S., & Haruechaiyasak, C. (2014). Cross-domain citation recommendation based on co-citation selection. In 11th International conference on electrical engineering/electronics, computer, telecommunications and information technology (ECTI-CON) (pp. 1–4). IEEE.

  • Torres, R., McNee, S. M., Abel, M., Konstan, J. A., & Riedl, J. (2004). Enhancing digital libraries with TechLens + . In Proceedings of the 4th ACM/IEEE-CS joint conference on digital libraries (pp. 228–236). ACM.

  • Totti, L. C., Mitra, P., Ouzzani, M., & Zaki, M. J. (2016). A query-oriented approach for relevance in citation networks. In Proceedings of the 25th international conference companion on world wide web (pp. 401–406). International World Wide Web Conferences Steering Committee.

  • Tsolakidis, A., Triperina, E., Sgouropoulou, C., & Christidis, N. (2016). Research publication recommendation system based on a hybrid approach. In Proceedings of the 20th Pan-Hellenic conference on informatics (pp. 78). ACM.

  • Uchiyama, K., Nanba, H., Aizawa, A., & Sagara, T. (2011). OSUSUME: Cross-lingual recommender system for research papers. In Proceedings of the 2011 workshop on context-awareness in retrieval and recommendation (pp. 39–42). ACM.

  • Verbert, K., Duval, E., Lindstaedt, S. N., & Gillet, D. (2010). Context-aware recommender systems. Journal of Universal Computer Science,16(16), 2175–2178.

    Google Scholar 

  • Voorhees, E. M. (2000). Variations in relevance judgments and the measurement of retrieval effectiveness. Information Processing and Management,36(5), 697–716.

    Article  Google Scholar 

  • Wan, Y. (2014). Exploring applying MeSH ontology for biomedical patent citation recommendation. Master's thesis. The Pennsylvania State University, University Park, PA, USA.

  • Wang, C., & Blei, D. M. (2011). Collaborative topic modeling for recommending scientific articles. In Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 448–456). ACM.

  • Wang, Q., Li, W., Zhang, X., & Lu, S. (2016). Academic paper recommendation based on community detection in citation-collaboration networks. In Asia-Pacific web conference (pp. 124–136). Springer.

  • Wang, J., Liao, L., Song, D., Ma, L., Lin, C.-Y., & Rui, Y. (2015a). Resorting relevance evidences to cumulative citation recommendation for knowledge base acceleration. In International conference on web-age information management (pp. 169–180). Springer.

  • Wang, J., Song, D., Wang, Q., Zhang, Z., Si, L., Liao, L., et al. (2015b). An entity class-dependent discriminative mixture model for cumulative citation recommendation. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval (pp. 635–644). ACM.

  • Wang, J., Song, D., Zhang, Z., Liao, L., Si, L., & Lin, C.-Y. (2015c). LDTM: A latent document type model for cumulative citation recommendation. In Proceedings of the 2015 conference on empirical methods in natural language processing (pp. 561–566).

  • Wang, Y., Wang, L., Li, Y., He, D., Chen, W., & Liu, T.-Y. (2013). A theoretical analysis of NDCG ranking measures. In Proceedings of the 26th annual conference on learning theory (COLT 2013) (Vol. 8, pp. 6).

  • Wesley-Smith, I., Dandrea, R. J., & West, J. D. (2015). An experimental platform for scholarly article recommendation. In BIR@ ECIR (pp. 30–39).

  • West, J. D., Wesley-Smith, I., & Bergstrom, C. T. (2016). A recommendation system based on hierarchical clustering of an article-level citation network. IEEE Transactions on Big Data,2(2), 113–123.

    Article  Google Scholar 

  • Weston, J., Bengio, S., & Usunier, N. (2010). Large scale image annotation: Learning to rank with joint word-image embeddings. Machine Learning,81(1), 21–35.

    Article  MathSciNet  Google Scholar 

  • Wu, H., Hua, Y., Li, B., & Pei, Y. (2012). Enhancing citation recommendation with various evidences. In 9th International conference on fuzzy systems and knowledge discovery (pp. 1160–1165). IEEE.

  • Xia, F., Liu, H., Lee, I., & Cao, L. (2016). Scientific article recommendation: Exploiting common author relations and historical preferences. IEEE Transactions on Big Data,2(2), 101–112.

    Article  Google Scholar 

  • Xie, C. (2015). An entity-centric query expansion approach to cumulative citation recommendation in knowledge base acceleration. In 12th International conference on fuzzy systems and knowledge discovery (FSKD) (pp. 1355–1359). IEEE.

  • Xue, H., Guo, J., Lan, Y., & Cao, L. (2014). Personalized paper recommendation in online social scholar system. In Proceedings of the 2014 IEEE/ACM international conference on advances in social networks analysis and mining (pp. 612–619). IEEE Press.

  • Yan, R., & Yan, H. (2013). Guess what you will cite: Personalized citation recommendation based on users’ preference. In Asia information retrieval symposium (pp. 428–439). Springer.

  • Yang, J., Yang, D., & Yang, X. (2013). A decision support system for finding research topic based on paper recommendation. In Proceedings of the Pacific Asia conference on Information systems(PACIS) (pp. 1–13).

  • Yang, Z., Yin, D., & Davison, B. D. (2014). Recommendation in academia: A joint multi-relational model. In IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM 2014) (pp. 566–571). IEEE.

  • Yilmaz, E., Kanoulas, E., & Aslam, J. A. (2008). A simple and efficient sampling method for estimating AP and NDCG. In Proceedings of the 31st annual international ACM SIGIR conference on research and development in information retrieval (pp. 603–610). ACM.

  • Yin, J., & Li, X. (2017). Personalized citation recommendation via convolutional neural networks. In Asia-Pacific web (APWeb) and web-age information management (WAIM) joint conference on web and big data (pp. 285–293). Springer.

  • Yin, Y., & Wang, D. (2017). The time dimension of science: Connecting the past to the future. Journal of Informetrics,11(2), 608–621.

    Article  Google Scholar 

  • Zarrinkalam, F., & Kahani, M. (2012). A multi-criteria hybrid citation recommendation system based on linked data. In 2nd International eConference on computer and knowledge engineering (ICCKE) (pp. 283–288). IEEE.

  • Zarrinkalam, F., & Kahani, M. (2013). SemCiR: A citation recommendation system based on a novel semantic distance measure. Program,47(1), 92–112.

    Article  Google Scholar 

  • Zhou, Q., Chen, X., & Chen, C. (2014). Authoritative scholarly paper recommendation based on paper communities. In IEEE 17th international conference on computational science and engineering (pp. 1536–1540). IEEE.

Download references

Acknowledgements

This work is supported by Major Projects of National Social Science Fund (No. 17ZDA291).

Author information

Authors and Affiliations

  1. Department of Information Management, Nanjing University of Science and Technology, Nanjing, China

    Shutian Ma & Chengzhi Zhang

  2. Department of Information and Library Science, Indiana University Bloomington, Bloomington, IN, USA

    Xiaozhong Liu

Authors
  1. Shutian Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chengzhi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaozhong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chengzhi Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, S., Zhang, C. & Liu, X. A review of citation recommendation: from textual content to enriched context. Scientometrics 122, 1445–1472 (2020). https://doi.org/10.1007/s11192-019-03336-0

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-019-03336-0

Keywords

Search

Navigation