short-paper

Adversarial Graph Perturbations for Recommendations at Scale

Authors:

Kwei-Herng Lai,

Hao YangAuthors Info & Claims

SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 1854 - 1858

https://doi.org/10.1145/3477495.3531763

Published: 07 July 2022 Publication History

Abstract

Graph Neural Networks (GNNs) provide a class of powerful architectures that are effective for graph-based collaborative filtering. Nevertheless, GNNs are known to be vulnerable to adversarial perturbations. Adversarial training is a simple yet effective way to improve the robustness of neural models. For example, many prior studies inject adversarial perturbations into either node features or hidden layers of GNNs. However, perturbing graph structures has been far less studied in recommendations.

To bridge this gap, we propose AdvGraph to model adversarial graph perturbations during the training of GNNs. Our AdvGraph is mainly based on min-max robust optimization, where an universal graph perturbation is obtained through an inner maximization while the outer optimization aims to compute the model parameters of GNNs. However, direct optimizing the inner problem is challenging due to discrete nature of the graph perturbations. To address this issue, an unbiased gradient estimator is further proposed to compute the gradients of discrete variables. Extensive experiments demonstrate that our AdvGraph is able to enhance the generalization performance of GNN-based recommenders.

Supplementary Material

MP4 File (SIGIR2022sp1238.mp4)

Video presentation for SIGIR 2022: Adversarial Graph Perturbations for Recommendations at Scale

Download
22.82 MB

References

[1]

George Casella and Christian P Robert. 1996. Rao-Blackwellisation of sampling schemes. Biometrika (1996), 81--94.

[2]

Huiyuan Chen and Jing Li. 2019. Adversarial tensor factorization for contextaware recommendation. In Proceedings of the 13th ACM Conference on Recommender Systems. 363--367.

Digital Library

[3]

Huiyuan Chen and Jing Li. 2019. Data poisoning attacks on cross-domain recommendation. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2177--2180.

Digital Library

[4]

Huiyuan Chen, Lan Wang, Yusan Lin, Chin-Chia Michael Yeh, Fei Wang, and Hao Yang. 2021. Structured graph convolutional networks with stochastic masks for recommender systems. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 614--623.

Digital Library

[5]

Hanjun Dai, Hui Li, Tian Tian, Xin Huang, Lin Wang, Jun Zhu, and Le Song. 2018. Adversarial attack on graph structured data. In International conference on machine learning. 1115--1124.

[6]

Zhijie Deng, Yinpeng Dong, and Jun Zhu. 2019. Batch virtual adversarial training for graph convolutional networks. arXiv preprint arXiv:1902.09192 (2019).

[7]

Alek Dimitriev and Mingyuan Zhou. 2021. ARMS: Antithetic-REINFORCE-Multi-Sample Gradient for Binary Variables. In International Conference on Machine Learning. 2717--2727.

[8]

Zhe Dong, Andriy Mnih, and George Tucker. 2020. DisARM: An Antithetic Gradient Estimator for Binary Latent Variables. In Advances in Neural Information Processing Systems.

[9]

Zhe Dong, Andriy Mnih, and George Tucker. 2021. Coupled Gradient Estimators for Discrete Latent Variables. In Advances in Neural Information Processing Systems.

[10]

Fuli Feng, Xiangnan He, Jie Tang, and Tat-Seng Chua. 2019. Graph adversarial training: Dynamically regularizing based on graph structure. IEEE Transactions on Knowledge and Data Engineering (2019).

[11]

Simon Geisler, Tobias Schmidt, Hakan ?irin, Daniel Zügner, Aleksandar Bojchevski, and Stephan Günnemann. 2021. Robustness of Graph Neural Networks at Scale. Advances in Neural Information Processing Systems (2021).

[12]

Arman Hasanzadeh, Ehsan Hajiramezanali, Shahin Boluki, Mingyuan Zhou, Nick Duffield, Krishna Narayanan, and Xiaoning Qian. 2020. Bayesian graph neural networks with adaptive connection sampling. In International conference on machine learning.

[13]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. 639--648.

Digital Library

[14]

Xiangnan He, Zhankui He, Xiaoyu Du, and Tat-Seng Chua. 2018. Adversarial personalized ranking for recommendation. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 355--364.

Digital Library

[15]

Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categorical reparameterization with gumbel-softmax. In International Conference on Learning Representations.

[16]

Hongwei Jin and Xinhua Zhang. 2019. Latent adversarial training of graph convolution networks. In ICML Workshop on Learning and Reasoning with Graph- Structured Representations.

[17]

Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. 2020. Graph structure learning for robust graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 66--74.

Digital Library

[18]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations.

[19]

Kezhi Kong, Guohao Li, Mucong Ding, Zuxuan Wu, Chen Zhu, Bernard Ghanem, Gavin Taylor, and Tom Goldstein. 2020. Flag: Adversarial data augmentation for graph neural networks. arXiv preprint arXiv:2010.09891 (2020).

[20]

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In International Conference on Learning Representations.

[21]

Kelong Mao, Jieming Zhu, Xi Xiao, Biao Lu, Zhaowei Wang, and Xiuqiang He. 2021. UltraGCN: Ultra Simplification of Graph Convolutional Networks for Recommendation. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 1253--1262.

Digital Library

[22]

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. 2017. Universal adversarial perturbations. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1765--1773.

[23]

Liang Qu, Huaisheng Zhu, Ruiqi Zheng, Yuhui Shi, and Hongzhi Yin. 2021. Im-GAGN: Imbalanced Network Embedding via Generative Adversarial Graph Networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1390--1398.

Digital Library

[24]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence. 452--461.

Digital Library

[25]

Ali Shafahi, Mahyar Najibi, Zheng Xu, John Dickerson, Larry S Davis, and Tom Goldstein. 2020. Universal adversarial training. In Proceedings of the AAAI Conference on Artificial Intelligence. 5636--5643.

[26]

Susheel Suresh, Pan Li, Cong Hao, and Jennifer Neville. 2021. Adversarial Graph Augmentation to Improve Graph Contrastive Learning. In Advances in Neural Information Processing Systems.

[27]

Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval. 165--174.

Digital Library

[28]

Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning 8, 3 (1992), 229--256.

[29]

Kaidi Xu, Hongge Chen, Sijia Liu, Pin-Yu Chen, Tsui-Wei Weng, Mingyi Hong, and Xue Lin. 2019. Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 3961--3967.

[30]

Mingzhang Yin and Mingyuan Zhou. 2019. ARM: Augment-REINFORCE-Merge Gradient for Stochastic Binary Networks. In International Conference on Learning Representations.

[31]

Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 974--983.

Digital Library

[32]

Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. 2019. GNNExplainer: Generating Explanations for Graph Neural Networks. In Advances in Neural Information Processing Systems.

[33]

Mingyuan Zhou and Lawrence Carin. 2013. Negative binomial process count and mixture modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence (2013), 307--320.

[34]

Dingyuan Zhu, Ziwei Zhang, Peng Cui, and Wenwu Zhu. 2019. Robust graph convolutional networks against adversarial attacks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1399--1407.

Digital Library

[35]

Daniel Zügner, Amir Akbarnejad, and Stephan Günnemann. 2018. Adversarial attacks on neural networks for graph data. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2847--2856.

Digital Library

[36]

Daniel Zügner and Stephan Günnemann. 2018. Adversarial Attacks on Graph Neural Networks via Meta Learning. In International Conference on Learning Representations.

Cited By

Miao RZhou KWang YLiu NWang YWang XSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Rethinking independent cross-entropy loss for graph-structured dataProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693518(35570-35589)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693518
Zhang JTang JChen XYu WHu LJiang PLi HBaeza-Yates RBonchi F(2024)Natural Language Explainable Recommendation with Robustness EnhancementProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671781(4203-4212)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671781
Wang JChen GXin KFei Z(2024)Metric learning with adversarial hard negative samples for tag recommendationThe Journal of Supercomputing10.1007/s11227-024-06274-880:14(21475-21507)Online publication date: 11-Jun-2024
https://dl.acm.org/doi/10.1007/s11227-024-06274-8
Show More Cited By

Index Terms

Adversarial Graph Perturbations for Recommendations at Scale

Recommendations

Graph Neural Patching for Cold-Start Recommendations
Databases Theory and Applications
Abstract
The cold start problem in recommender systems remains a critical challenge. Current solutions often train hybrid models on auxiliary data for both cold and warm users/items, potentially degrading the experience for the latter. This drawback limits ...
Linear-Time Graph Neural Networks for Scalable Recommendations
WWW '24: Proceedings of the ACM Web Conference 2024

In an era of information explosion, recommender systems are vital tools to deliver personalized recommendations for users. The key of recommender systems is to forecast users' future behaviors based on previous user-item interactions. Due to their strong ...
Adversarial Training on Weights for Graph Neural Networks
ACAI '22: Proceedings of the 2022 5th International Conference on Algorithms, Computing and Artificial Intelligence

Despite the fact that Graph Neural Networks (GNNs) have been extensively used for graph embedding representation, it is challenging to train well-performing GNNs on graphs with good generalization due to the limitation of overfitting. Previous research ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2022

3569 pages

ISBN:9781450387323

DOI:10.1145/3477495

General Chairs:
Enrique Amigo
UNED
,
Pablo Castells
UAM and Amazon
,
Julio Gonzalo
UNED
,
Program Chairs:
Ben Carterette
Spotify
,
J. Shane Culpepper
RMIT University
,
Gabriella Kazai
Waseda University

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

SIGIR '22

Sponsor:

SIGIR

SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2022

Madrid, Spain

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
339
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)3

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Miao RZhou KWang YLiu NWang YWang XSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Rethinking independent cross-entropy loss for graph-structured dataProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693518(35570-35589)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693518
Zhang JTang JChen XYu WHu LJiang PLi HBaeza-Yates RBonchi F(2024)Natural Language Explainable Recommendation with Robustness EnhancementProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671781(4203-4212)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671781
Wang JChen GXin KFei Z(2024)Metric learning with adversarial hard negative samples for tag recommendationThe Journal of Supercomputing10.1007/s11227-024-06274-880:14(21475-21507)Online publication date: 11-Jun-2024
https://dl.acm.org/doi/10.1007/s11227-024-06274-8
Lai VChen HYeh CXu MCai YYang H(2023)Enhancing Transformers without Self-supervised Learning: A Loss Landscape Perspective in Sequential RecommendationProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608831(791-797)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608831
Chen HZhou KLai KYeh CZheng YHu XYang H(2023)Hessian-aware Quantized Node Embeddings for RecommendationProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608826(757-762)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608826
Chen HLi XLai VYeh CFan YZheng YDas MYang H(2023)Adversarial Collaborative Filtering for FreeProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608771(245-255)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608771
Lai KZha DChen HBendre MChen YDas MYang HHu XFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Tackling Diverse Minorities in Imbalanced ClassificationProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615071(1178-1187)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615071
Liu ZFang YWu M(2023)Mitigating Popularity Bias for Users and Items with Fairness-centric Adaptive RecommendationACM Transactions on Information Systems10.1145/356428641:3(1-27)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1145/3564286
Chen HYeh CFan YZheng YWang JLai VDas MYang HChen HDuh WHuang HKato MMothe JPoblete B(2023)Sharpness-Aware Graph Collaborative FilteringProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3592059(2369-2373)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3592059
Merra FAnelli VDi Noia TMalitesta DMancino AChen HDuh WHuang HKato MMothe JPoblete B(2023)Denoise to Protect: A Method to Robustify Visual Recommenders from AdversariesProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591971(1924-1928)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591971
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten