skip to main content
10.1145/3477495.3531770acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
short-paper

EFLEC: Efficient Feature-LEakage Correction in GNN based Recommendation Systems

Published: 07 July 2022 Publication History

Abstract

Graph Convolutional Neural Networks (GNN) based recommender systems are state-of-the-art since they can capture the high order collaborative signals between users and items. However, they suffer from the feature leakage problem since label information determined by edges can be leaked into node embeddings through the GNN aggregation procedure guided by the same set of edges, leading to poor generalization. We propose the accurate removal algorithm to generate the final embedding. For each edge, the embeddings of the two end nodes are evaluated on a graph with that edge removed. We devise an algebraic trick to efficiently compute this procedure without explicitly constructing separate graphs for the LightGCN model. Experiments on four datasets demonstrate that our algorithm can perform better on datasets with sparse interactions, while the training time is significantly reduced.

Supplementary Material

MP4 File (SIGIR22-sp1288.mp4)
We elaborate on the issue of feature leakage problem in the training procedure of GNN-based recommender systems. We demonstrate the advantages and disadvantages of two motivative solutions and present our main idea of combining their strengths of them.

References

[1]
Rianne van den Berg, Thomas N Kipf, and Max Welling. 2017. Graph convolutional matrix completion. arXiv preprint arXiv:1706.02263 (2017).
[2]
Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2013. Spectral networks and locally connected networks on graphs. In Proc. Int. Conf. Learning Representations.
[3]
Soumen Chakrabarti, Earl Cox, Eibe Frank, Ralf Hartmut Güting, Jiawei Han, Xia Jiang, Micheline Kamber, Sam S Lightstone, Thomas P Nadeau, Richard E Neapolitan, et al. 2008. Data mining: know it all. Morgan Kaufmann.
[4]
Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics. JMLRWorkshop and Conference Proceedings, 249--256.
[5]
Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: A Factorization-Machine based Neural Network for CTR Prediction. In Proc. Int. Joint Conf. Artificial Intelligence.
[6]
William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, Vol. 2017-Decem. 1025--1035. arXiv:1706.02216
[7]
Ruining He and Julian McAuley. 2016. Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering. In Proceedings of the 25th International Conference on World Wide Web (Montréal, Québec, Canada) (WWW '16). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 507--517. https://doi.org/10.1145/2872427.2883037
[8]
Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. 639--648.
[9]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web. 173--182.
[10]
Thomas Hofmann. 2004. Latent semantic models for collaborative filtering. ACM Trans. Information System (2004).
[11]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[12]
Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In Proc. Int. Conf. Learning Representations.
[13]
Yehuda Koren. 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proc. ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining.
[14]
Yehuda Koren, Robert M. Bell, and Chris Volinsky. 2009. Matrix Factorization Techniques for Recommender Systems. IEEE Computer (2009).
[15]
Chong Li, Kunyang Jia, Dan Shen, C.J. Richard Shi, and Hongxia Yang. 2019. Hierarchical Representation Learning for Bipartite Graphs. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 2873--2879. https://doi.org/10.24963/ijcai.2019/398
[16]
Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, and Guangzhong Sun. 2018. xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems. In Proc. ACM SIGKDD Int. Conf. Knowledge Discovery & Data Mining.
[17]
Bin Liu, Ruiming Tang, Yingzhi Chen, Jinkai Yu, Huifeng Guo, and Yuzhou Zhang. 2019. Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction. In Proc. World Wide Web Conference.
[18]
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. 2015. Image-Based Recommendations on Styles and Substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (Santiago, Chile) (SIGIR '15). Association for Computing Machinery, New York, NY, USA, 43--52. https://doi.org/10.1145/2766462.2767755
[19]
Yanru Qu, Bohui Fang, Weinan Zhang, Ruiming Tang, Minzhe Niu, Huifeng Guo, Yong Yu, and Xiuqiang He. 2018. Product-Based Neural Networks for User Response Prediction over Multi-Field Categorical Data. ACM Trans. Inf. Syst. 37, 1, Article 5 (Oct. 2018), 35 pages. https://doi.org/10.1145/3233770
[20]
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In Proc. Conf. Uncertainty in Artificial Intelligence.
[21]
Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. 2020. DropEdge: Towards Deep Graph Convolutional Networks on Node Classification. In International Conference on Learning Representations. https://openreview.net/forum?id=Hkx1qkrKPr
[22]
Jianing Sun, Wei Guo, Dengcheng Zhang, Yingxue Zhang, Florence Regol, Yaochen Hu, Huifeng Guo, Ruiming Tang, Han Yuan, Xiuqiang He, et al. 2020. A framework for recommending accurate and diverse items using bayesian graph convolutional neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2030--2039.
[23]
Jianing Sun, Yingxue Zhang,Wei Guo, Huifeng Guo, Ruiming Tang, Xiuqiang He, Chen Ma, and Mark Coates. 2020. Neighbor Interaction Aware Graph Convolution Networks for Recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, China) (SIGIR '20). Association for Computing Machinery, New York, NY, USA, 1289--1298. https://doi.org/10.1145/3397271.3401123
[24]
Jianing Sun, Yingxue Zhang, Chen Ma, Mark Coates, Huifeng Guo, Ruiming Tang, and Xiuqiang He. 2019. Multi-graph convolution collaborative filtering. In 2019 IEEE International Conference on Data Mining (ICDM). IEEE, 1306--1311.
[25]
Qiaoyu Tan, Ninghao Liu, Xing Zhao, Hongxia Yang, Jingren Zhou, and Xia Hu. 2020. Learning to Hash with Graph Neural Networks for Recommender Systems. In Proceedings of The Web Conference 2020 (WWW '20). Association for Computing Machinery, New York, NY, USA, 1988--1998. https://doi.org/10.1145/3366423.3380266
[26]
Haoyu Wang, Defu Lian, and Yong Ge. 2019. Binarized Collaborative Filtering with Distilling Graph Convolutional Network. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 4802--4808. https://doi.org/10.24963/ijcai.2019/667
[27]
Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat Seng Chua. 2019. Neural graph collaborative filtering. SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (2019), 165--174.
[28]
Xiang Wang, Hongye Jin, An Zhang, Xiangnan He, Tong Xu, and Tat-Seng Chua. 2020. Disentangled Graph Collaborative Filtering. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, China) (SIGIR '20). Association for Computing Machinery, New York, NY, USA, 1001--1010. https://doi.org/10.1145/3397271.3401137
[29]
Xiao Wang, Ruijia Wang, Chuan Shi, Guojie Song, and Qingyong Li. 2020. Multicomponent graph convolutional collaborative filtering. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 6267--6274.
[30]
Shiwen Wu, Wentao Zhang, Fei Sun, and Bin Cui. 2020. Graph Neural Networks in Recommender Systems: A Survey. 37, 4 (2020). http://arxiv.org/abs/2011.02260
[31]
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2018), 974--983.
[32]
Jiani Zhang, Xingjian Shi, Shenglin Zhao, and Irwin King. 2019. STAR-GCN: Stacked and reconstructed graph convolutional networks for recommender systems. IJCAI International Joint Conference on Artificial Intelligence 2019-Augus (2019), 4264--4270. https://doi.org/10.24963/ijcai.2019/592
[33]
Muhan Zhang and Yixin Chen. 2018. Link Prediction Based on Graph Neural Networks. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2018/file/53f0d7c537d99b3824f0f99d62ea2428-Paper.pdf

Cited By

View all
  • (2024)Quantum Nearest Neighbor Collaborative Filtering Algorithm for Recommendation SystemACM Transactions on Knowledge Discovery from Data10.1145/367498218:8(1-28)Online publication date: 29-Jun-2024
  • (2024)Cluster-driven Personalized Federated Recommendation with Interest-aware Graph Convolution Network for MultimediaProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680788(5614-5622)Online publication date: 28-Oct-2024
  • (2024)AKGNN: Attribute Knowledge Graph Neural Networks Recommendation for Corporate Volunteer ActivitiesIEEE Transactions on Big Data10.1109/TBDATA.2024.345376110:6(720-730)Online publication date: Dec-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval
July 2022
3569 pages
ISBN:9781450387323
DOI:10.1145/3477495
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. feature leakage correction
  2. graph neural networks
  3. recommendation systems

Qualifiers

  • Short-paper

Conference

SIGIR '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)28
  • Downloads (Last 6 weeks)3
Reflects downloads up to 28 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Quantum Nearest Neighbor Collaborative Filtering Algorithm for Recommendation SystemACM Transactions on Knowledge Discovery from Data10.1145/367498218:8(1-28)Online publication date: 29-Jun-2024
  • (2024)Cluster-driven Personalized Federated Recommendation with Interest-aware Graph Convolution Network for MultimediaProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680788(5614-5622)Online publication date: 28-Oct-2024
  • (2024)AKGNN: Attribute Knowledge Graph Neural Networks Recommendation for Corporate Volunteer ActivitiesIEEE Transactions on Big Data10.1109/TBDATA.2024.345376110:6(720-730)Online publication date: Dec-2024
  • (2023)DAG: Dual Attention Graph Representation Learning for Node ClassificationMathematics10.3390/math1117369111:17(3691)Online publication date: 28-Aug-2023
  • (2023)Algorithm/Hardware Co-Optimization for Sparsity-Aware SpMM Acceleration of GNNsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2023.328171442:12(4763-4776)Online publication date: 31-May-2023
  • (2023)A surrogate evolutionary neural architecture search algorithm for graph neural networksApplied Soft Computing10.1016/j.asoc.2023.110485144:COnline publication date: 1-Sep-2023
  • (2023)Graphs get personal: learning representation with contextual pretraining for collaborative filteringApplied Intelligence10.1007/s10489-023-05144-953:24(30416-30430)Online publication date: 17-Nov-2023
  • (2022)IMGC-GNN: A multi-granularity coupled graph neural network recommendation method based on implicit relationshipsApplied Intelligence10.1007/s10489-022-04215-753:11(14668-14689)Online publication date: 1-Nov-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media