Skip to main content
SpringerLink
Log in

SPR-SMN: scientific paper recommendation employing SPECTER with memory network

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

During the last decades, recommender systems are becoming quite popular since they provide great assistance to users on social networks and library websites. Unfortunately, the large volume of data combined with sparsity makes personalization a difficult task. In this regard, several models were introduced in the literature that suffers from the cold-start problem and the lack of personalization. In particular, the majority of these models ignore the relationship between the important factors and the semantic relations among the nodes (the authors, and the field of study) on the heterogeneous papers networks. Moreover, they fail to effectively capture researchers’ preferences, which leads to inadequate recommendations. To overcome these problems, with this study we propose a scientific paper recommendation model called SPR-SMN, which employs the SPECTER document embedding model to learn context-preserving paper content representations. The model captures the long-range dependencies and researchers’ preferences, by employing an end-to-end memory network and personalization module, respectively. We experimentally evaluate our method against baseline algorithms over two real-life datasets. The results indicate that the proposed method outperforms competing models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. https://www.aminer.cn/citation.

  2. https://acl-arc.comp.nus.edu.sg/.

References

  • Abro, W. A., Qi, G., Ali, Z., Feng, Y., & Aamir, M. (2020). Multi-turn intent determination and slot filling with neural networks and regular expressions. Knowledge-Based Systems, 208, 106428.

    Article  Google Scholar 

  • Achakulvisut, T., Acuna, D. E., Ruangrong, T., & Kording, K. (2016). Science concierge: A fast content-based recommendation system for scientific publications. PLoS ONE, 11(7), 1–11.

    Article  Google Scholar 

  • Ali, Z., Kefalas, P., Muhammad, K., Ali, B., & Imran, M. (2020a). Deep learning in citation recommendation models survey. Expert Systems with Applications, 162, 113790.

    Article  Google Scholar 

  • Ali, Z., Qi, G., Kefalas, P., Abro, W. A., & Ali, B. (2020b). A graph-based taxonomy of citation recommendation models. Artificial Intelligence Review, 53(7), 5217–5260.

    Article  Google Scholar 

  • Ali, Z., Qi, G., Muhammad, K., Ali, B., & Abro, W. A. (2020c). Paper recommendation based on heterogeneous network embedding. Knowledge-Based Systems, 210, 106438.

    Article  Google Scholar 

  • Ali, Z., Qi, G., Muhammad, K., Bhattacharyya, S., Ullah, I., & Abro, W. A. (2021a). Citation recommendation employing heterogeneous bibliographic network embedding. Neural Computing and Applications. https://doi.org/10.1007/s00521-021-06135-y

    Article  Google Scholar 

  • Ali, Z., Qi, G., Muhammad, K., Kefalas, P., & Khusro, S. (2021b). Global citation recommendation employing generative adversarial network. Expert Systems with Applications., 180, 114888.

    Article  Google Scholar 

  • Ali, Z., Ullah, I., Khan, A., Ullah Jan, A., & Muhammad, K. (2021c). An overview and evaluation of citation recommendation models. Scientometrics, 126(5), 4083–4119.

    Article  Google Scholar 

  • Amami, M., Pasi, G., Stella, F., & Faiz, R. (2016). An LDA-based approach to scientific paper recommendation. In: Natural Language Processing and Information Systems: Proceedings in 21st International Conference on Applications of Natural Language to Information Systems, NLDB 2016, Salford, UK, June 22-24, pp. 200–210

  • Amir, N., Jabeen, F., Ali, Z., Ullah, I., Jan, A. U., & Kefalas, P. (2022). On the current state of deep learning for news recommendation. Artificial Intelligence Review. https://doi.org/10.1007/s10462-022-10191-8

    Article  Google Scholar 

  • Bansal, T., Belanger, D., McCallum, A. (2016). Ask the gru: Multi-task learning for deep text recommendations. In: Proceedings of the 10th ACM Conference on Recommender Systems (RecSys), RecSys ’16, pp. 107–114. New York, NY, USA

  • Beltagy, I., Lo, K., & Cohan, A. (2019). Scibert: A pretrained language model for scientific text.

  • Bhagavatula, C., Feldman, S., Power, R., & Ammar, W. (2018). Content-based citation recommendation. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics (pp. 238–251). New Orleans: Human Language Technologies.

    Google Scholar 

  • Bollacker, K., Lawrence, S., & Giles, C. L. (1998). Citeseer: an autonomous web agent for automatic retrieval and identification of interesting publications. In: Proceedings of the second international conference on Autonomous agents, pp. 116–123

  • Cai, X., Han, J., Li, W., Zhang, R., Pan, S., & Yang, L. (2018a). A three-layered mutually reinforced model for personalized citation recommendation. Transactions on Neural Networks and Learning Systems, 29(12), 6026–6037.

    Article  Google Scholar 

  • Cai, X., Han, J., & Yang, L. (2018b). Generative adversarial network based heterogeneous bibliographic network representation for personalized citation recommendation. In: Thirty-Second AAAI Conference on Artificial Intelligence

  • Cai, X., Zheng, Y., Yang, L., Dai, T., & Guo, L. (2019). Bibliographic network representation based personalized citation recommendation. IEEE Access, 7, 457–467.

    Article  Google Scholar 

  • Chen, J., Liu, Y., Zhao, S., & Zhang, Y. (2019). Citation recommendation based on weighted heterogeneous information network containing semantic linking. 2019 IEEE International Conference on Multimedia and Expo (ICME) (pp. 31–36). Shanghai: IEEE.

    Chapter  Google Scholar 

  • Chen, Y. L., Yeh, Y. H., & Ma, M. R. (2021). A movie recommendation method based on users’ positive and negative profiles. Information Processing and Management, 58, 102531.

    Article  Google Scholar 

  • Christoforidis, G., Kefalas, P., Papadopoulos, A., & Manolopoulos, Y. (2018). Recommendation of points-of-interest using graph embeddings. In: 5th IEEE International Conference on Data Science and Advanced Analytics (DSAA), 2018, Turin, Italy, October 1-3, 2018, pp. 31–40

  • Christoforidis, G., Kefalas, P., Papadopoulos, A. N., & Manolopoulos, Y. (2021a). Reline: point-of-interest recommendations using multiple network embeddings. Knowledge and Information Systems, 63(4), 791–817.

    Article  Google Scholar 

  • Christoforidis, G., Kefalas, P., Papadopoulos, A. N., & Manolopoulos, Y. (2021b). RELINE: point-of-interest recommendations using multiple network embeddings. Knowledge and Information Systems, 63(4), 791–817.

    Article  Google Scholar 

  • Cohan, A., Feldman, S., Beltagy, I., Downey, D., & Weld, D. (2020). Specter: Document-level representation learning using citation-informed transformers. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 1–13). Association for Computational Linguistics.

    Google Scholar 

  • Cui, P., Wang, X., Pei, J., & Zhu, W. (2019). A survey on network embedding. IEEE Transactions on Knowledge and Data Engineering, 31(5), 833–852.

    Article  Google Scholar 

  • Dai, T., Yan, W., Zhang, K., Qiu, C., Zhao, X., & Pan, S. (2021). Gated relational stacked denoising autoencoder with localized author embedding for global citation recommendation. Expert Systems with Applications, 184, 115359.

    Article  Google Scholar 

  • Dai, T., Zhu, L., Wang, Y., & Carley, K. M. (2019). Attentive stacked denoising autoencoder with bi-lstm for personalized context-aware citation recommendation. Transactions on Audio, Speech, and Language Processing (TACL), 28, 553–568.

    Article  Google Scholar 

  • Devlin, J., Chang, M.W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805

  • Du, N., Guo, J., Wu, C. Q., Hou, A., Zhao, Z., & Gan, D. (2020). Recommendation of academic papers based on heterogeneous information networks. 2020 IEEE/ACS 17th International Conference on Computer Systems and Applications (AICCSA) (pp. 1–6). IEEE.

    Google Scholar 

  • Du, Z., Tang, J., & Ding, Y. (2019). Polar: Attention-based cnn for one-shot personalized article recommendation. Machine Learning and Knowledge Discovery in Databases (pp. 675–690). Springer.

    Chapter  Google Scholar 

  • Ganguly, S., & Pudi, V. (2017). Paper2vec: Combining graph and text information for scientific paper representation. Advances in Information Retrieval (pp. 383–395). Cham: Springer.

    Chapter  Google Scholar 

  • Gao, Z., Feng, A., Song, X., & Wu, X. (2019). Target-dependent sentiment classification with bert. IEEE Access, 7, 154290–154299.

    Article  Google Scholar 

  • Goyal, P., & Ferrara, E. (2018). Graph embedding techniques, applications, and performance: A survey. Knowledge-Based Systems, 151, 78–94.

    Article  Google Scholar 

  • Grover, A., Leskovec, J. (2016). Node2vec: Scalable feature learning for networks. In: Proceedings of the 22Nd ACM International Conference on Knowledge Discovery and Data Mining (KDD), pp. 855–864

  • Guo, L., Cai, X., Qin, H., Guo, Y., Li, F., Tian, G. (2019). Citation recommendation with a content-sensitive deepwalk based approach. In: International Conference on Data Mining Workshops (ICDMW), pp. 538–543

  • Gupta, S., Varma, V. (2017). Scientific article recommendation by using distributed representations of text and graph. In: Proceedings of the 26th International Conference on World Wide Web Companion, WWW ’17 Companion, pp. 1267–1268.

  • Huang, W., Wu, Z., Liang, C., Mitra, P., Giles, C. L. (2015). A neural probabilistic model for context based citation recommendation. In: Proceedings of the Twenty-Ninth Conference on Artificial Intelligence (AAAI), pp. 2404–2410

  • Jiang, Z., Yin, Y., Gao, L., Lu, Y., Liu, X. (2018). Cross-language citation recommendation via hierarchical representation learning on heterogeneous graph. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR ’18, p. 635-644

  • Kefalas, P., & Manolopoulos, Y. (2017). A time-aware spatio-textual recommender system. Expert Systems with Applications, 78, 396–406.

    Article  Google Scholar 

  • Kefalas, P., & Symeonidis, P. (2015). Recommending friends and locations over a heterogeneous spatio-temporal graph. MEDI. Springer.

    Google Scholar 

  • Kefalas, P., Symeonidis, P., & Manolopoulos, Y. (2018). Recommendations based on a heterogeneous spatio-temporal social network. World Wide Web, 21(2), 345–371.

    Article  Google Scholar 

  • Khadka, A., Cantador, I., Fernández, M.: Exploiting citation knowledge in personalised recommendation of recent scientific publications. In: LREC (2020)

  • Khusro, S., Ali, Z., & Ullah, I. (2016). Recommender systems: Issues, challenges, and research opportunities. Information Science and Applications (ICISA) 2016 (pp. 1179–1189). Springer.

    Chapter  Google Scholar 

  • Kong, X., Mao, M., Wang, W., Liu, J., & Xu, B. (2019). Voprec: Vector representation learning of papers with text information and structural identity for recommendation. Transactions on Emerging Topics in Computing (TETC) (pp. 1–12). IEEE.

    Google Scholar 

  • Kreutz, C.K., Schenkel, R.: Scientific paper recommendation systems: a literature review of recent publications. ArXiv abs/2201.00682 (2022)

  • Le, Q., & Mikolov, T. (2014). Distributed representations of sentences and documents. Proceedings of the 31st International Conference on Machine Learning (pp. 1188–1196). PLMR.

    Google Scholar 

  • Ma, Y., Mao, J., Ba, Z., & Li, G. (2020). Location recommendation by combining geographical, categorical, and social preferences with location popularity. Information Processing and Management, 57, 102251.

    Article  Google Scholar 

  • McNee, S.M., Albert, I., Cosley, D., Gopalkrishnan, P., Lam, S.K., Rashid, A.M., Konstan, J.A., Riedl, J. (2002). On the recommending of citations for research papers. In: Proceedings of the 2002 ACM conference on Computer supported cooperative work, pp. 116–125

  • Perozzi, B., Al-Rfou, R., Skiena, S. (2014). Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, pp. 701–710

  • Qiu, T., Yu, C., Zhong, Y., An, L., & Li, G. (2021). A scientific citation recommendation model integrating network and text representations. Scientometrics, 126(11), 9199–9221.

    Article  Google Scholar 

  • Reimers, N., Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. In: EMNLP/IJCNLP

  • Ribeiro, L.F., Saverese, P.H., Figueiredo, D.R. (2017). struc2vec: Learning node representations from structural identity. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 385–394

  • Salloum, S., & Rajamanthri, D. (2021). Implementation and evaluation of movie recommender systems using collaborative filtering. Journal of Advances in Information Technology. https://doi.org/10.12720/jait.12.3.189-196

    Article  Google Scholar 

  • Son, J., & Kim, S. B. (2017). Academic paper recommender system using multilevel simultaneous citation networks. Decision Support Systems, 105, 24–33.

    Article  Google Scholar 

  • Sukhbaatar, S., Szlam, A., Weston, J., Fergus, R.: End-to-end memory networks. arXiv preprint arXiv:1503.08895 (2015)

  • Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q. (2015). Line: Large-scale information network embedding. In: Proceedings of the 24th International Conference on World Wide Web (WWW), pp. 1067–1077

  • Uddin, M. N., Li, B., Ali, Z., Kefalas, P., Khan, I., & Zada, I. (2022). Software defect prediction employing bilstm and bert-based semantic feature. Soft Computing. https://doi.org/10.1007/s00500-022-06830-5

    Article  Google Scholar 

  • Wang, C., Blei, D.M. (2011). Collaborative topic modeling for recommending scientific articles. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pp. 448–456

  • Wang, H., & Li, W. (2015). Relational collaborative topic regression for recommender systems. Transactions on Knowledge and Data Engineering (TKDE), 27(5), 1343–1355.

    Article  MathSciNet  Google Scholar 

  • Wang, J., Zhu, L., Dai, T., & Wang, Y. (2020a). Deep memory network with bi-lstm for personalized context-aware citation recommendation. Neurocomputing, 410, 103–113.

    Article  Google Scholar 

  • Wang, L., Rao, Y., Bian, Q., & Wang, S. (2020b). Content-based hybrid deep neural network citation recommendation method. International Conference of Pioneering Computer Scientists, Engineers and Educators (pp. 3–20). Springer.

    Google Scholar 

  • Yang, L., Zheng, Y., Cai, X., Dai, H., Mu, D., Guo, L., & Dai, T. (2018). A lstm based model for personalized context-aware citation recommendation. IEEE Access, 6, 59618–59627.

    Article  Google Scholar 

  • Yin, J., & Li, X. (2017). Personalized citation recommendation via convolutional neural networks. Asia-Pacific web (APWeb) and web-age information management (WAIM) joint conference on web and big data (pp. 285–293). Springer.

    Google Scholar 

  • Zhang, J., & Zhu, L. (2021). Citation recommendation using semantic representation of cited papers’ relations and content. Expert Systems with Applications, 187, 115826.

    Article  Google Scholar 

  • Zhao, W., Wu, R., & Liu, H. (2016). Paper recommendation based on the knowledge gap between a researcher’s background knowledge and research target. Information Processing and Management, 52, 976–988.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Southeast University, Nanjing, China

    Zafar Ali & Guilin Qi

  2. Department of Informatics, Aristotle University, Thessaloniki, Greece

    Pavlos Kefalas

  3. Department of Computer Science, University of Peshawar, Peshawar, Pakistan

    Shah Khusro

  4. Department of Computer Science, University of Buner, Buner, Pakistan

    Inayat Khan

  5. Visual Analytics for Knowledge Laboratory (VIS2KNOW Lab), Department of Applied Artificial Intelligence, School of Convergence, College of Computing and Informatics, Sungkyunkwan University, Seoul, 03063, Republic of Korea

    Khan Muhammad

Authors
  1. Zafar Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guilin Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pavlos Kefalas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shah Khusro
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Inayat Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Khan Muhammad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zafar Ali.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interest or personal relationships that could have appeared to influence the work reported in this paper.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ali, Z., Qi, G., Kefalas, P. et al. SPR-SMN: scientific paper recommendation employing SPECTER with memory network. Scientometrics 127, 6763–6785 (2022). https://doi.org/10.1007/s11192-022-04425-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-022-04425-3

Keywords

Search

Navigation