research-article

MixGCF: An Improved Training Method for Graph Neural Network-based Recommender Systems

Authors:

Jie TangAuthors Info & Claims

KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining

Pages 665 - 674

https://doi.org/10.1145/3447548.3467408

Published: 14 August 2021 Publication History

Abstract

Graph neural networks (GNNs) have recently emerged as state-of-the-art collaborative filtering (CF) solution. A fundamental challenge of CF is to distill negative signals from the implicit feedback, but negative sampling in GNN-based CF has been largely unexplored. In this work, we propose to study negative sampling by leveraging both the user-item graph structure and GNNs' aggregation process. We present the MixGCF method---a general negative sampling plugin that can be directly used to train GNN-based recommender systems. In MixGCF, rather than sampling raw negatives from data, we design the hop mixing technique to synthesize hard negatives. Specifically, the idea of hop mixing is to generate the synthetic negative by aggregating embeddings from different layers of raw negatives' neighborhoods. The layer and neighborhood selection process are optimized by a theoretically-backed hard selection strategy. Extensive experiments demonstrate that by using MixGCF, state-of-the-art GNN-based recommendation models can be consistently and significantly improved, e.g., 26% for NGCF and 22% for LightGCN in terms of NDCG@20.

Supplementary Material

MP4 File (mixgcf_an_improved_training_method-tinglin_huang-yuxiao_dong-38957982-Q6f0.mp4)

Presentation Video on KDD2021: MixGCF: An Improved Training Method for Graph Neural Network-based Recommender Systems

Download
86.75 MB

References

[1]

Liwei Cai and William Yang Wangc. 2018. KBGAN: Adversarial Learning for Knowledge Graph Embeddings. In NAACL-HLT'18. 1470--1480.

[2]

Yixin Cao, Xiang Wang, Xiangnan He, Zikun Hu, and Tat-Seng Chua. 2019. Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences. In WWW.

[3]

Lei Chen, Le Wu, Richang Hong, Kun Zhang, and Meng Wang. 2020. Revisiting Graph based Collaborative Filtering: A Linear Residual Graph Convolutional Network Approach. In AAAI.

[4]

Ting Chen, Yizhou Sun, Yue Shi, and Liangjie Hong. 2017. On Sampling Strategies for Neural Network-based Collaborative Filtering. In KDD. 767--776.

[5]

Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep Neural Networks for YouTube Recommendations. In RecSys. 191--198.

[6]

Jingtao Ding, Yuhan Quan, Quanming Yao, Yong Li, and Depeng Jin. 2020. Simplify and Robustify Negative Sampling for Implicit Collaborative Filtering. In NeurIPS.

[7]

Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph neural networks for social recommendation. In WWW. 417--426.

[8]

Wenzheng Feng, Jie Zhang, Yuxiao Dong, Yu Han, Huanbo Luan, Qian Xu, Qiang Yang, Evgeny Kharlamov, and Jie Tang. 2020. Graph Random Neural Network for Semi-Supervised Learning on Graphs. In NeurIPS.

[9]

Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In AISTATS. 249--256.

[10]

Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In KDD. 855--864.

[11]

William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In NeurIPS. 1025--1035.

[12]

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In CVPR.

[13]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. In SIGIR.

[14]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural Collaborative Filtering. In WWW. 173--182.

[15]

Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, and Joaquin Quiñonero Candela. 2014. Practical Lessons from Predicting Clicks on Ads at Facebook. In ADKDD.

[16]

Cheng-Kang Hsieh, Longqi Yang, Yin Cui, Tsung-Yi Lin, Serge Belongie, and Deborah Estrin. 2017. Collaborative Metric Learning. In WWW.

[17]

Jui-Ting Huang, Ashish Sharma, Shuying Sun, Li Xia, David Zhang, Philip Pronin, Janani Padmanabhan, Giuseppe Ottaviano, and Linjun Yang. 2020. Embeddingbased Retrieval in Facebook Search. In KDD.

[18]

Yannis Kalantidis, Mert Bulent Sariyildiz, Noe Pion, Philippe Weinzaepfel, and Diane Larlus. 2020. Hard Negative Mixing for Contrastive Learning. In NeurIPS.

[19]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.

[20]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.

[21]

Yehuda Koren, Robert M. Bell, and Chris Volinsky. 2009. Matrix Factorization Techniques for Recommender Systems. In IEEE Computer. 42, 8 (2009), 30--37.

Digital Library

[22]

Walid Krichene and Steffen Rendle. 2020. On Sampled Metrics for Item Recommendation. In KDD. 1748--1757.

[23]

Hao Ma, Haixuan Yang, Michael R Lyu, and Irwin King. 2008. Sorec: social recommendation using probabilistic matrix factorization. In CIKM. 931--940.

[24]

Hao Ma, Dengyong Zhou, Chao Liu, Michael R Lyu, and Irwin King. 2011. Recommender systems with social regularization. In WSDM. 287--296.

[25]

Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van DenHengel. 2015. Image-based recommendations on styles and substitutes. In SIGIR.

[26]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In NeurIPS. 3111--3119.

[27]

Rong Pan, Yunhong Zhou, Bin Cao, Nathan N Liu, Rajan Lukose, Martin Scholz, and Qiang Yang. 2008. One-class collaborative filtering. In ICDM. 502--511.

[28]

Dae Hoon Park and Yi Chang. 2019. Adversarial Sampling and Training for Semi-Supervised Information Retrieval. In WWW. 1443--1453.

[29]

Jiezhong Qiu, Jian Tang, Hao Ma, Yuxiao Dong, KuansanWang, and Jie Tang. 2018. DeepInf: Social Influence Prediction with Deep Learning. In KDD. 2110--2119.

[30]

Steffen Rendle and Christoph Freudenthaler. 2014. Improving pairwise learning for item recommendation from implicit feedback. In WSDM. 273--282.

[31]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In UAI. 452--461.

Digital Library

[32]

Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. FaceNet: A Unified Embedding for Face Recognition and Clustering. In CVPR.

[33]

Patrice Y. Simard, Yann A. Le Cun, John S. Denker, and Bernard Victorri. 1998. Transformation invariance in pattern recognitiontangent distance and tangent propagation. In Neural networks: tricks of the trade.

[34]

Kihyuk Sohn. 2016. Improved Deep Metric Learning with Multi-class N-pair Loss Objective. In NeurIPS.

[35]

Qiaoyu Tan, Ninghao Liu, Xing Zhao, Hongxia Yang, Jingren Zhou, and Xia Hu. 2020. Learning to Hash with Graph Neural Networks for Recommender Systems. In WWW. 1988--1998.

[36]

Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, JunYan, and Qiaozhu Mei. 2015. Line: Large-scale information network embedding. In WWW. 1067--1077.

Digital Library

[37]

Rianne van den Berg, Thomas N Kipf, and Max Welling. 2018. Graph Convolutional Matrix Completion. In KDD.

[38]

Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2018. RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems. In CIKM. 417--426.

[39]

HongweiWang, Fuzheng Zhang, Mengdi Zhang, Jure Leskovec, Miao Zhao,Wenjie Li, and Zhongyuan Wang. 2019. Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems. In KDD.

[40]

Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, Peng Zhang, and Dell Zhang. 2017. IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models. In SIGIR. 515--524.

Digital Library

[41]

Qinyong Wang, Hongzhi Yin, Zhiting Hu, Defu Lian, Hao Wang, and Zi Huang. 2018. Neural Memory Streaming Recommender Networks with Adversarial Training. In KDD. 2467--2475.

[42]

Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019. KGAT: Knowledge Graph Attention Network for Recommendation. In KDD. 950--958.

Digital Library

[43]

Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural Graph Collaborative Filtering. In SIGIR.

[44]

Xiang Wang, Yaokun Xu, Xiangnan He, Yixin Cao, Meng Wang, and Tat-Seng Chua. 2020. Reinforced Negative Sampling over Knowledge Graph for Recommendation. In WWW.

[45]

Ze Wang, Guangyan Lin, Huobin Tan, Qinghong Chen, and Xiyang Liu. 2020. CKAN: Collaborative Knowledge-aware Attentive Network for Recommender Systems. In SIGIR. 219--228.

Digital Library

[46]

Le Wu, Peijie Sun, Yanjie Fu, Richang Hong, Xiting Wang, and Meng Wang. 2019. A Neural Influence Diffusion Model for Social Recommendation. In SIGIR.

[47]

Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken ichi Kawarabayashi, and Stefanie Jegelka. 2018. Representation Learning on Graphs with Jumping Knowledge Networks. In ICML.

[48]

Zhen Yang, Ming Ding, Chang Zhou, Hongxia Yang, Jingren Zhou, and Jie Tang. 2020. Understanding Negative Sampling in Graph Representation Learning. In KDD.

[49]

Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In KDD. 974--983.

[50]

Hao Yu, Huanyu Wang, and Jianxin Wu. 2021. Mixup Without Hesitation. arXiv preprint arXiv:2101.04342 (2021).

[51]

Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. 2018. mixup: Beyond empirical risk minimization. In ICLR.

[52]

Weinan Zhang, Tianqi Chen, Jun Wang, and Yong Yu. 2013. Optimizing top-n collaborative filtering via dynamic negative item sampling. In SIGIR. 785--788.

[53]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In KDD. 1059--1068.

Cited By

Wang YGe XChen XXie RYan SZhang XChen ZMa JXin XNejdl WAuer SKarras OCha MMoens MNajork M(2025)Exploration and Exploitation of Hard Negative Samples for Cross-Domain Sequential RecommendationProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining10.1145/3701551.3703535(669-677)Online publication date: 10-Mar-2025
https://dl.acm.org/doi/10.1145/3701551.3703535
Lee CChoi JWi HCho SPark NNejdl WAuer SKarras OCha MMoens MNajork M(2025)SCONE: A Novel Stochastic Sampling to Generate Contrastive Views and Hard Negative Samples for RecommendationProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining10.1145/3701551.3703522(419-428)Online publication date: 10-Mar-2025
https://dl.acm.org/doi/10.1145/3701551.3703522
Fan YLi CGe JHuang LLuo B(2025)Effective Hard Negative Mining for Contrastive Learning-Based Code SearchACM Transactions on Software Engineering and Methodology10.1145/369599434:3(1-35)Online publication date: 23-Feb-2025
https://dl.acm.org/doi/10.1145/3695994
Show More Cited By

Index Terms

MixGCF: An Improved Training Method for Graph Neural Network-based Recommender Systems
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems
2. Mathematics of computing
  1. Discrete mathematics
    1. Graph theory
      1. Graph algorithms

Recommendations

A Scalable, Accurate Hybrid Recommender System
WKDD '10: Proceedings of the 2010 Third International Conference on Knowledge Discovery and Data Mining

Recommender systems apply machine learning techniques for filtering unseen information and can predict whether a user would like a given resource. There are three main types of recommender systems: collaborative filtering, content-based filtering, and ...
SCONE: A Novel Stochastic Sampling to Generate Contrastive Views and Hard Negative Samples for Recommendation
WSDM '25: Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining

Graph-based collaborative filtering (CF) has emerged as a promising approach in recommender systems. Despite its achievements, graph-based CF models face challenges due to data sparsity and negative sampling. In this paper, we propose a novel Stochastic ...
A framework for diversifying recommendation lists by user interest expansion

Recommender systems have been widely used to discover users' preferences and recommend interesting items to users during this age of information overload. Researchers in the field of recommender systems have realized that the quality of a top-N ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining

August 2021

4259 pages

ISBN:9781450383325

DOI:10.1145/3447548

General Chairs:
Feida Zhu
Singapore Management University
,
Beng Chin Ooi
National University of Singapore
,
Chunyan Miao
Nanyang Technology University
,
Program Chairs:
Haixun Wang,
Iryna Skrypnyk,
Wynne Hsu,
Sanjay Chawla

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation for Distinguished Young Scholars
National Natural ScienceFoundation of China Key Program

Conference

KDD '21

Sponsor:

KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 14 - 18, 2021

Virtual Event, Singapore

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

121
Total Citations
View Citations
1,975
Total Downloads

Downloads (Last 12 months)342
Downloads (Last 6 weeks)25

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang YGe XChen XXie RYan SZhang XChen ZMa JXin XNejdl WAuer SKarras OCha MMoens MNajork M(2025)Exploration and Exploitation of Hard Negative Samples for Cross-Domain Sequential RecommendationProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining10.1145/3701551.3703535(669-677)Online publication date: 10-Mar-2025
https://dl.acm.org/doi/10.1145/3701551.3703535
Lee CChoi JWi HCho SPark NNejdl WAuer SKarras OCha MMoens MNajork M(2025)SCONE: A Novel Stochastic Sampling to Generate Contrastive Views and Hard Negative Samples for RecommendationProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining10.1145/3701551.3703522(419-428)Online publication date: 10-Mar-2025
https://dl.acm.org/doi/10.1145/3701551.3703522
Fan YLi CGe JHuang LLuo B(2025)Effective Hard Negative Mining for Contrastive Learning-Based Code SearchACM Transactions on Software Engineering and Methodology10.1145/369599434:3(1-35)Online publication date: 23-Feb-2025
https://dl.acm.org/doi/10.1145/3695994
Qayyum AIrnawan BXu SHu ZFukumoto FSuzuki Y(2025)Textual Graph Contrastive Learning for Enhanced Dataset Recommendation2025 19th International Conference on Ubiquitous Information Management and Communication (IMCOM)10.1109/IMCOM64595.2025.10857584(1-4)Online publication date: 3-Jan-2025
https://doi.org/10.1109/IMCOM64595.2025.10857584
Yang FDu HZhang XYang YWang Y(2025)Self-supervised category-enhanced graph neural networks for recommendationKnowledge-Based Systems10.1016/j.knosys.2025.113109311(113109)Online publication date: Feb-2025
https://doi.org/10.1016/j.knosys.2025.113109
Zhang KSang GCheng JLiu ZZhang Y(2025)Negative sampling strategy based on multi-hop neighbors for graph representation learningExpert Systems with Applications10.1016/j.eswa.2024.125688263(125688)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125688
Wani SKhan SQuadri S(2025)Application of Deep Learning for Single Cell Multi-Omics: A State-of-the-Art ReviewArchives of Computational Methods in Engineering10.1007/s11831-025-10230-xOnline publication date: 1-Mar-2025
https://doi.org/10.1007/s11831-025-10230-x
Li YZhao MZhang JXie ZLiu YZhan Q(2025)GDDRec: graph neural diffusion model for diversified recommendationKnowledge and Information Systems10.1007/s10115-025-02348-yOnline publication date: 1-Feb-2025
https://doi.org/10.1007/s10115-025-02348-y
Chen BLi HShi Z(2024)Research on Spurious-Negative Sample Augmentation-Based Quality Evaluation Method for Cybersecurity Knowledge GraphMathematics10.3390/math1301006813:1(68)Online publication date: 27-Dec-2024
https://doi.org/10.3390/math13010068
Duan ZWang CZhong W(2024)SSGCL: Simple Social Recommendation with Graph Contrastive LearningMathematics10.3390/math1207110712:7(1107)Online publication date: 7-Apr-2024
https://doi.org/10.3390/math12071107
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten