skip to main content
10.1145/3640457.3691703acmconferencesArticle/Chapter ViewAbstractPublication PagesrecsysConference Proceedingsconference-collections
extended-abstract

KGGLM: A Generative Language Model for Generalizable Knowledge Graph Representation Learning in Recommendation

Published: 08 October 2024 Publication History

Abstract

Current recommendation methods based on knowledge graphs rely on entity and relation representations for several steps along the pipeline, with knowledge completion and path reasoning being the most influential. Despite their similarities, the most effective representation methods for these steps differ, leading to inefficiencies, limited representativeness, and reduced interpretability. In this paper, we introduce KGGLM, a decoder-only Transformer model designed for generalizable knowledge representation learning to support recommendation. The model is trained on generic paths sampled from the knowledge graph to capture foundational patterns, and then fine-tuned on paths specific of the downstream step (knowledge completion and path reasoning in our case). Experiments on ML1M and LFM1M show that KGGLM beats twenty-two baselines in effectiveness under both knowledge completion and recommendation. Source code and pre-processed data sets are available at https://github.com/mirkomarras/kgglm.

References

[1]
Qingyao Ai, Vahid Azizi, Xu Chen, and Yongfeng Zhang. 2018. Learning Heterogeneous Knowledge Base Embeddings for Explainable Recommendation. Algorithms 11, 9 (2018), 137.
[2]
Ivana Balazevic, Carl Allen, and Timothy Hospedales. 2019. TuckER: Tensor Factorization for Knowledge Graph Completion. In Proc. of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing (EMNLP-IJCNLP 2019). ACM.
[3]
Giacomo Balloccu, Ludovico Boratto, Christian Cancedda, Gianni Fenu, and Mirko Marras. 2023. Knowledge is Power, Understanding is Impact: Utility and Beyond Goals, Explanation Quality, and Fairness in Path Reasoning Recommendation. In Proc. of the 45th European Conference on Information Retrieval (ECIR 2023), Vol. 13982. Springer, 3–19.
[4]
Giacomo Balloccu, Ludovico Boratto, Gianni Fenu, and Mirko Marras. 2022. Post Processing Recommender Systems with Knowledge Graphs for Recency, Popularity, and Diversity of Explanations. In Proc. of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2022. ACM, 646–656.
[5]
Giacomo Balloccu, Ludovico Boratto, Gianni Fenu, and Mirko Marras. 2023. Reinforcement recommendation reasoning through knowledge graphs for explanation path quality. Knowl. Based Syst. 260 (2023), 110098.
[6]
Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In Proc. of the 27th Annual Conference on Neural Information Processing Systems (NIPS 2013). Curran Associates, Inc., 2787–2795.
[7]
Yixin Cao, Xiang Wang, Xiangnan He, Zikun Hu, and Tat-Seng Chua. 2019. Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences. In Proc. of the The World Wide Web Conference (WWW 2019). ACM.
[8]
Zhe Chen, Yuehan Wang, Bin Zhao, Jing Cheng, Xin Zhao, and Zongtao Duan. 2020. Knowledge Graph Completion: A Review. IEEE Access 8 (2020), 192435–192456.
[9]
Danilo Dessì, Mauro Dragoni, Gianni Fenu, Mirko Marras, and Diego Reforgiato Recupero. 2019. Evaluating neural word embeddings created from online course reviews for sentiment analysis. In Proc. of the 34th ACM/SIGAPP Symposium on Applied Computing, SAC 2019. ACM, 2124–2127.
[10]
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2D Knowledge Graph Embeddings. In Proc. of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18). AAAI Press, 1811–1818.
[11]
Takuma Ebisu and Ryutaro Ichise. 2018. TorusE: Knowledge Graph Embedding on a Lie Group. In Proc. of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), Sheila A. McIlraith and Kilian Q. Weinberger (Eds.). AAAI Press, 1819–1826.
[12]
Shijie Geng, Zuohui Fu, Juntao Tan, Yingqiang Ge, Gerard de Melo, and Yongfeng Zhang. 2022. Path Language Modeling over Knowledge Graphs for Explainable Recommendation. In Proc. of the ACM Web Conference 2022 WWW 2022). ACM, 946–955.
[13]
F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems 5, 19 (2015), 19.
[14]
Subhi Issa, Onaopepo Adekunle, Fayçal Hamdi, Samira Si-Said Cherfi, Michel Dumontier, and Amrapali Zaveri. 2021. Knowledge Graph Completeness: A Systematic Literature Review. IEEE Access 9 (2021), 31322–31339.
[15]
Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Knowledge Graph Embedding via Dynamic Mapping Matrix. In Proc. of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2015). ACM, 687–696.
[16]
Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning Entity and Relation Embeddings for Knowledge Graph Completion. In Proc. of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI 2015)(AAAI’15). AAAI Press, 2181–2187.
[17]
Hanxiao Liu, Yuexin Wu, and Yiming Yang. 2017. Analogical Inference for Multi-relational Embeddings. In Proc. of the 34th International Conference on Machine Learning (ICML 2017), Doina Precup and Yee Whye Teh (Eds.). Vol. 70. PMLR, 2168–2178.
[18]
Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Phung. 2018. A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network. In Proc. of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2018). ACL, 1–8.
[19]
Maximilian Nickel, Lorenzo Rosasco, and Tomaso A. Poggio. 2016. Holographic Embeddings of Knowledge Graphs. In Proc. of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI 2016). AAAI Press, 1955–1961.
[20]
Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A three-way model for collective learning on multi-relational data. In Proc. of the 28th International Conference on International Conference on Machine Learning (ICML 2011)(ICML’11). Omnipress, Madison, WI, USA, 809–816.
[21]
Thamizhiniyan Pugazhenthi and Huizhi Liang. 2022. Improving Conversational Recommender Systems via Knowledge Graph-based Semantic Fusion with Historical Interaction Data. In Proc. of the IEEE International Conference on Big Data, Big Data 2022. IEEE, 4303–4312.
[22]
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. (2019).
[23]
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language Models are Unsupervised Multitask Learners. OpenAI Blog 1, 8 (2019), 9.
[24]
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In Proc. of the 25th Conference on Uncertainty in Artificial Intelligence (UAI 2009). AUAI Press, 452–461.
[25]
Amina Samih, Abderrahim Ghadi, and Abdelhadi Fennan. 2020. Translational-randomwalk embeddings-based recommender systems: a pragmatic survey. 957–966.
[26]
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR abs/1910.01108 (2019). arXiv:1910.01108
[27]
Markus Schedl. 2016. The LFM-1b Dataset for Music Retrieval and Recommendation. In Proc. of the 2016 ACM on International Conference on Multimedia Retrieval (ICMR 2016). ACM, 103–110.
[28]
Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197 (2019).
[29]
Chang-You Tai, Liang-Ying Huang, Chien-Kun Huang, and Lun-Wei Ku. 2021. User-Centric Path Reasoning towards Explainable Recommendation. In Proc. of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021). Association for Computing Machinery, 879–889.
[30]
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. LLaMA: Open and Efficient Foundation Language Models. CoRR abs/2302.13971 (2023).
[31]
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. [n. d.]. Complex Embeddings for Simple Link Prediction. In Proc. of the 33nd International Conference on Machine Learning (ICML 2016), volume = 48, pages = 2071–2080, publisher = JMLR.org, year = 2016, timestamp = Wed, 29 May 2019 08:41:46 +0200, biburl = https://dblp.org/rec/conf/icml/TrouillonWRGB16.bib, bibsource = dblp computer science bibliography, https://dblp.org, Maria-Florina Balcan and Kilian Q. Weinberger (Eds.).
[32]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. In Proc. of the Annual Conference on Neural Information Processing Systems (NIPS 2017). 5998–6008.
[33]
Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019. KGAT: Knowledge Graph Attention Network for Recommendation. In Proc. of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (SIGKDD 2019).
[34]
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge Graph Embedding by Translating on Hyperplanes. Proc. of the AAAI Conference on Artificial Intelligence (AAAI 2014) 28, 1 (2014), 1–8.
[35]
Yikun Xian, Zuohui Fu, S. Muthukrishnan, Gerard de Melo, and Yongfeng Zhang. 2019. Reinforcement Knowledge Graph Reasoning for Explainable Recommendation. In Proc. of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019). 285–294.
[36]
Yikun Xian, Zuohui Fu, Handong Zhao, Yingqiang Ge, Xu Chen, Qiaoying Huang, Shijie Geng, Zhou Qin, Gerard de Melo, S. Muthukrishnan, and Yongfeng Zhang. 2020. CAFE: Coarse-to-Fine Neural Symbolic Reasoning for Explainable Recommendation. In Proc. of the 29th ACM International Conference on Information & Knowledge Management (CIKM 2020).
[37]
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In Prco. of the 3rd International Conference on Learning Representations (ICLR 2015), Yoshua Bengio and Yann LeCun (Eds.).
[38]
Yuhao Yang, Chao Huang, Lianghao Xia, and Chenliang Li. 2022. Knowledge Graph Contrastive Learning for Recommendation. In Proc. of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2022. ACM, 1434–1443.
[39]
Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. 2016. Collaborative Knowledge Base Embedding for Recommender Systems. In Proc. of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (SIGKDD 2016). 353–362.
[40]
Jiajun Zhou, Bo Liu, Lu Deng, Yaofeng Chen, and Zhefeng Xiao. 2018. On Random Walk Based Weighted Graph Sampling. IEICE Trans. Inf. Syst. 101-D, 2 (2018), 535–538.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
RecSys '24: Proceedings of the 18th ACM Conference on Recommender Systems
October 2024
1438 pages
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 October 2024

Check for updates

Author Tags

  1. Generative Artificial Intelligence.
  2. Knowledge Completion
  3. Knowledge Graph
  4. Knowledge Graph Embeddings
  5. Knowledge Representation Learning
  6. Language Model
  7. Recommendation

Qualifiers

  • Extended-abstract
  • Research
  • Refereed limited

Funding Sources

  • eINS Ecosystem of Innovation for Next Generation Sardinia, National Recovery and Resilience Plan (NRRP), Miss. 4 Comp. 2 Inv. 1.5 - Call for tender No.3277 published on Dec 30, 2021 by the Italian Ministry of University and Research (MUR) funded by the European Union ? NextGenerationEU

Conference

Acceptance Rates

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 266
    Total Downloads
  • Downloads (Last 12 months)266
  • Downloads (Last 6 weeks)38
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media