skip to main content
10.1145/3397271.3401040acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

Jointly Non-Sampling Learning for Knowledge Graph Enhanced Recommendation

Published: 25 July 2020 Publication History

Abstract

Knowledge graph (KG) contains well-structured external information and has shown to be effective for high-quality recommendation. However, existing KG enhanced recommendation methods have largely focused on exploring advanced neural network architectures to better investigate the structural information of KG. While for model learning, these methods mainly rely on Negative Sampling (NS) to optimize the models for both KG embedding task and recommendation task. Since NS is not robust (e.g., sampling a small fraction of negative instances may lose lots of useful information), it is reasonable to argue that these methods are insufficient to capture collaborative information among users, items, and entities.
In this paper, we propose a novel Jointly Non-Sampling learning model for Knowledge graph enhanced Recommendation (JNSKR). Specifically, we first design a new efficient NS optimization algorithm for knowledge graph embedding learning. The subgraphs are then encoded by the proposed attentive neural network to better characterize user preference over items. Through novel designs of memorization strategies and joint learning framework, JNSKR not only models the fine-grained connections among users, items, and entities, but also efficiently learns model parameters from the whole training data (including all non-observed data) with a rather low time complexity. Experimental results on two public benchmarks show that JNSKR significantly outperforms the state-of-the-art methods like RippleNet and KGAT. Remarkably, JNSKR also shows significant advantages in training efficiency (about 20 times faster than KGAT), which makes it more applicable to real-world large-scale systems.

Supplementary Material

MP4 File (3397271.3401040.mp4)
Presentation Video of the paper "Jointly Non-Sampling Learning for Knowledge Graph Enhanced Recommendation". The source code of JNSKR and datasets used in the paper have been made available: "https://github.com/chenchongthu/JNSKR"

References

[1]
Qingyao Ai, Vahid Azizi, Xu Chen, and Yongfeng Zhang. 2018. Learning heterogeneous knowledge base embeddings for explainable recommendation. Algorithms, Vol. 11, 9 (2018), 137.
[2]
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Proceedings of NeuIPS. 2787--2795.
[3]
Yixin Cao, Xiang Wang, Xiangnan He, Zikun Hu, and Tat-Seng Chua. 2019. Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences. In Proceedings of WWW. 151--161.
[4]
Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. 2018. Neural Attentional Rating Regression with Review-level Explanations. In Proceedings of WWW.
[5]
Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. 2019 a. Social Attentional Memory Network: Modeling Aspect-and Friend-level Differences in Recommendation. In Proceedings of WSDM.
[6]
Chong Chen, Min Zhang, Weizhi Ma, Yiqun Liu, and Shaoping Ma. 2020 a. Efficient Non-Sampling Factorization Machines for Optimal Context-Aware Recommendation. In Proceedings of WWW. 2400--2410.
[7]
Chong Chen, Min Zhang, Chenyang Wang, Weizhi Ma, Minming Li, Yiqun Liu, and Shaoping Ma. 2019 b. An Efficient Adaptive Transfer Neural Network for Social-aware Recommendation. In Proceedings of SIGIR. ACM, 225--234.
[8]
Chong Chen, Min Zhang, Yongfeng Zhang, Yiqun Liu, and Shaoping Ma. 2020 b. Efficient Neural Matrix Factorization without Sampling for Recommendation. ACM Trans. Inf. Syst., Vol. 38, 2, Article Article 14 (Jan. 2020).
[9]
Chong Chen, Min Zhang, Yongfeng Zhang, Weizhi Ma, Yiqun Liu, and Shaoping Ma. 2020 c. Efficient Heterogeneous Collaborative Filtering without Negative Sampling for Recommendation. In Proceedings of AAAI.
[10]
Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. 2017. Attentive collaborative filtering: Multimedia recommendation with item-and component-level attention. In Proceedings of SIGIR. 335--344.
[11]
John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, Vol. 12, Jul (2011), 2121--2159.
[12]
Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of WWW. 507--517.
[13]
Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of SIGIR. 355--364.
[14]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of WWW. 173--182.
[15]
Xiangnan He, Hanwang Zhang, Min-Yen Kan, and Tat-Seng Chua. 2016. Fast matrix factorization for online recommendation with implicit feedback. In Proceedings of SIGIR. 549--558.
[16]
Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In Proceedings of ICDM. 263--272.
[17]
Jin Huang, Wayne Xin Zhao, Hongjian Dou, Ji-Rong Wen, and Edward Y Chang. 2018. Improving sequential recommendation with knowledge-enhanced memory networks. In Proceedings of SIGIR. ACM, 505--514.
[18]
Yehuda Koren. 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of KDD. 426--434.
[19]
Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 8 (2009), 30--37.
[20]
Dawen Liang, Laurent Charlin, James McInerney, and David M Blei. 2016. Modeling user exposure in recommendation. In Proceedings of WWW. 951--961.
[21]
Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of AAAI.
[22]
Weizhi Ma, Min Zhang, Yue Cao, Woojeong Jin, Chenyang Wang, Yiqun Liu, Shaoping Ma, and Xiang Ren. 2019. Jointly Learning Explainable Rules for Recommendation with Knowledge Graph. In Proceedings of WWW.
[23]
Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of ICML. 807--814.
[24]
István Pilászy, Dávid Zibriczky, and Domonkos Tikk. 2010. Fast als-based matrix factorization for explicit and implicit feedback datasets. In Proceedings of Recsys.
[25]
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of UAI. 452--461.
[26]
Francesco Ricci, Lior Rokach, and Bracha Shapira. 2011. Introduction to recommender systems handbook. In Recommender systems handbook. Springer.
[27]
Roy Schwartz, Jesse Dodge, Noah A Smith, and Oren Etzioni. 2019. Green AI. arXiv preprint arXiv:1907.10597 (2019).
[28]
Yu Shi, Qi Zhu, Fang Guo, Chao Zhang, and Jiawei Han. 2018. Easing Embedding Learning by Comprehensive Transcription of Heterogeneous Information Networks. In Proceedings of KDD. 2190--2199.
[29]
Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, Vol. 15, 1 (2014), 1929--1958.
[30]
Zhu Sun, Jie Yang, Jie Zhang, Alessandro Bozzon, Long-Kai Huang, and Chi Xu. 2018. Recurrent knowledge graph embedding for effective recommendation. In Proceedings of RecSys. ACM, 297--305.
[31]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of NeuIPS. 5998--6008.
[32]
Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In ICLR.
[33]
Maksims Volkovs, Guangwei Yu, and Tomi Poutanen. 2017. Dropoutnet: Addressing cold start in recommender systems. In Proceedings of NeuIPS. 4957--4966.
[34]
Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2018. Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In Proceedings of CIKM. 417--426.
[35]
Hongwei Wang, Fuzheng Zhang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2019 c. Multi-Task Feature Learning for Knowledge Graph Enhanced Recommendation. In Proceedings of WWW. ACM, 2000--2010.
[36]
Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, and Minyi Guo. 2019 d. Knowledge graph convolutional networks for recommender systems. In Proceedings of WWW. ACM, 3307--3313.
[37]
Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. 2017. Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering, Vol. 29, 12 (2017), 2724--2743.
[38]
Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019 a. KGAT: Knowledge Graph Attention Network for Recommendation. In Proceedings of KDD.
[39]
Xiang Wang, Dingxian Wang, Canran Xu, Xiangnan He, Yixin Cao, and Tat-Seng Chua. 2019 b. Explainable reasoning over knowledge graphs for recommendation. In Proceedings of AAAI, Vol. 33. 5329--5336.
[40]
Yikun Xian, Zuohui Fu, S Muthukrishnan, Gerard De Melo, and Yongfeng Zhang. 2019. Reinforcement knowledge graph reasoning for explainable recommendation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 285--294.
[41]
Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua. 2017. Attentional factorization machines: Learning the weight of feature interactions via attention networks. arXiv preprint arXiv:1708.04617 (2017).
[42]
Xin Xin, Xiangnan He, Yongfeng Zhang, Yongdong Zhang, and Joemon Jose. 2019. Relational Collaborative Filtering: Modeling Multiple Item Relations for Recommendation. In Proceedings of SIGIR.
[43]
Xin Xin, Fajie Yuan, Xiangnan He, and Joemon M Jose. 2018. Batch IS NOT Heavy: LearningWord Representations From All Samples. (2018).
[44]
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2014. Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575 (2014).
[45]
Xiao Yu, Xiang Ren, Yizhou Sun, Quanquan Gu, Bradley Sturt, Urvashi Khandelwal, Brandon Norick, and Jiawei Han. 2014. Personalized entity recommendation: A heterogeneous information network approach. In Proceedings of WSDM.
[46]
Fajie Yuan, Xin Xin, Xiangnan He, Guibing Guo, Weinan Zhang, Chua Tat-Seng, and Joemon M Jose. 2018. fbgd: Learning embeddings from positive unlabeled data with bgd. (2018).
[47]
Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. 2016. Collaborative knowledge base embedding for recommender systems. In Proceedings of KDD. 353--362.
[48]
Yongfeng Zhang, Xu Chen, et almbox. 2020. Explainable Recommendation: A Survey and New Perspectives. Foundations and Trends® in Information Retrieval, Vol. 14, 1 (2020), 1--101.
[49]
Huan Zhao, Quanming Yao, Jianda Li, Yangqiu Song, and Dik Lun Lee. 2017. Meta-graph based recommendation fusion over heterogeneous information networks. In Proceedings of KDD. ACM, 635--644.

Cited By

View all
  • (2024)InLN: Knowledge-aware Incremental Leveling Network for Dynamic AdvertisingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3672032(1679-1690)Online publication date: 25-Aug-2024
  • (2024)Interpretable Disease Progression Prediction Based on Reinforcement Reasoning Over a Knowledge GraphIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2023.333184754:3(1948-1959)Online publication date: Mar-2024
  • (2024)Does Negative Sampling Matter? a Review With Insights Into its Theory and ApplicationsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.337147346:8(5692-5711)Online publication date: Aug-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval
July 2020
2548 pages
ISBN:9781450380164
DOI:10.1145/3397271
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. efficient
  2. implicit feedback
  3. knowledge graph
  4. non-sampling learning
  5. recommender systems

Qualifiers

  • Research-article

Funding Sources

  • Natural Science Foundation of China
  • National Key Research and Development Program of China

Conference

SIGIR '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)55
  • Downloads (Last 6 weeks)6
Reflects downloads up to 27 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)InLN: Knowledge-aware Incremental Leveling Network for Dynamic AdvertisingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3672032(1679-1690)Online publication date: 25-Aug-2024
  • (2024)Interpretable Disease Progression Prediction Based on Reinforcement Reasoning Over a Knowledge GraphIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2023.333184754:3(1948-1959)Online publication date: Mar-2024
  • (2024)Does Negative Sampling Matter? a Review With Insights Into its Theory and ApplicationsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.337147346:8(5692-5711)Online publication date: Aug-2024
  • (2024)Knowledge Graph-Based Behavior Denoising and Preference Learning for Sequential RecommendationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.332566636:6(2490-2503)Online publication date: Jun-2024
  • (2024)Knowledge-aware Recommendation combining Intent Networks with counterfactual Generators2024 5th International Conference on Computer Engineering and Application (ICCEA)10.1109/ICCEA62105.2024.10603817(544-548)Online publication date: 12-Apr-2024
  • (2024)Recent advances on federated learning: A systematic surveyNeurocomputing10.1016/j.neucom.2024.128019597(128019)Online publication date: Sep-2024
  • (2024)An attention mechanism and residual network based knowledge graph-enhanced recommender systemKnowledge-Based Systems10.1016/j.knosys.2024.112042299(112042)Online publication date: Sep-2024
  • (2023)NSDIF: Leveraging Non-Sampling Learning and Denoising Implicit Feedback for RecommendationProceedings of the 2023 5th International Conference on Internet of Things, Automation and Artificial Intelligence10.1145/3653081.3653129(289-294)Online publication date: 24-Nov-2023
  • (2023)SelfCF: A Simple Framework for Self-supervised Collaborative FilteringACM Transactions on Recommender Systems10.1145/35914691:2(1-25)Online publication date: 12-Jun-2023
  • (2023)Reinforced PU-learning with Hybrid Negative Sampling Strategies for RecommendationACM Transactions on Intelligent Systems and Technology10.1145/358256214:3(1-25)Online publication date: 8-May-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media