skip to main content
research-article

Training Robust Deep Collaborative Filtering Models via Adversarial Noise Propagation

Published: 18 August 2023 Publication History

Abstract

The recommendation performance of deep collaborative filtering models drops sharply under imperceptible adversarial perturbations. Some methods promote the robustness of recommendation systems by adversarial training. However, these methods only study shallow models and lack the exploration of deep models. Furthermore, the way these methods add adversarial noise to the weight parameters of users and items is not fully applicable to deep collaborative filtering models, because the adversarial noise is not sufficient to fully affect its network structure with multiple hidden layers. In this article, we propose a novel adversarial training framework, Random Layer-wise Adversarial Training (RAT), which trains a robust deep collaborative filtering model via adversarial noise propagation. Specifically, we inject adversarial noise into the output of the hidden layer in a random layer-wise manner. The adversarial noise propagates forward from the injected position to obtain more flexible model parameters during the adversarial training process. We validate the effectiveness of RAT on multilayer perceptron (MLP) and implement RAT on MLP-based and convolutional neural networks-based deep collaborative filtering models. Experiments on three publicly available datasets show that the deep collaborative filtering model trained by RAT not only defends against adversarial noise but also guarantees recommendation performance.

References

[1]
Vito Walter Anelli, Yashar Deldjoo, Tommaso Di Noia, Daniele Malitesta, and Felice Antonio Merra. 2021. A study of defensive methods to protect visual recommendation against adversarial manipulation of images. In SIGIR. 1094–1103.
[2]
Vito Walter Anelli, Tommaso Di Noia, and Felice Antonio Merra. 2021. The idiosyncratic effects of adversarial training on bias in personalized recommendation learning. In RecSys. 730–735.
[3]
Hai Chen, Fulan Qian, Jie Chen, Shu Zhao, and Yanping Zhang. 2021. Attribute-based neural collaborative filtering. Expert Syst. Applic. 185 (2021), 115539.
[4]
Hai Chen, Fulan Qian, Jie Chen, Shu Zhao, and Yanping Zhang. 2021. FG-RS: Capture user fine-grained preferences through attribute information for recommender systems. Neurocomputing 458 (2021), 195–203.
[5]
Ming Chen and Xiuze Zhou. 2020. DeepRank: Learning to rank with neural networks for recommendation. Knowl-based Syst. 209 (2020), 106478.
[6]
Rami Cohen, Oren Sar Shalom, Dietmar Jannach, and Amihood Amir. 2021. A black-box attack model for visually-aware recommender systems. In WSDM. 94–102.
[7]
Nilaksh Das, Madhuri Shanbhogue, Shang-Tse Chen, Fred Hohman, Li Chen, Michael E. Kounavis, and Duen Horng Chau. 2017. Keeping the bad guys out: Protecting and vaccinating deep learning with JPEG compression. arXiv preprint arXiv:1705.02900 (2017).
[8]
Zhi-Hong Deng, Ling Huang, Chang-Dong Wang, Jian-Huang Lai, and S. Yu Philip. 2019. DeepCF: A unified framework of representation learning and matching function learning in recommender system. In AAAI, Vol. 33. 61–68.
[9]
Tommaso Di Noia, Daniele Malitesta, and Felice Antonio Merra. 2020. TAAMR: Targeted adversarial attack against multimedia recommender systems. In IEEE/IFIP. IEEE, 1–8.
[10]
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. 2018. Boosting adversarial attacks with momentum. In CVPR. 9185–9193.
[11]
Y. Du, M. Fang, J. Yi, C. Xu, J. Cheng, and D. Tao. 2018. Enhancing the robustness of neural collaborative filtering systems under malicious attacks. Trans Multim. 21, 3 (2018), 555–565.
[12]
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).
[13]
Ruining He and Julian McAuley. 2016. VBPR: visual Bayesian personalized ranking from implicit feedback. In AAAI, Vol. 30.
[14]
Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and powering graph convolution network for recommendation. In SIGIR. ACM, 639–648.
[15]
Xiangnan He, Xiaoyu Du, Xiang Wang, Feng Tian, Jinhui Tang, and Tat-Seng Chua. 2018. Outer product-based neural collaborative filtering. arXiv preprint arXiv:1808.03912 (2018).
[16]
Xiangnan He, Zhankui He, Xiaoyu Du, and Tat-Seng Chua. 2018. Adversarial personalized ranking for recommendation. In SIGIR. ACM, 355–364.
[17]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In WWW. 173–182.
[18]
Xiangnan He, Hanwang Zhang, Min-Yen Kan, and Tat-Seng Chua. 2016. Fast matrix factorization for online recommendation with implicit feedback. In SIGIR. 549–558.
[19]
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. 2019. Using pre-training can improve model robustness and uncertainty. In ICML. PMLR, 2712–2721.
[20]
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. 2018. Black-box adversarial attacks with limited queries and information. In ICML. PMLR, 2137–2146.
[21]
Donghyun Kim, Chanyoung Park, Jinoh Oh, Sungyoung Lee, and Hwanjo Yu. 2016. Convolutional matrix factorization for document context-aware recommendation. In RecSys. 233–240.
[22]
Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37.
[23]
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2018. Adversarial examples in the physical world. In Artificial intelligence safety and security. Chapman and Hall/CRC, 99–112.
[24]
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521, 7553 (2015), 436–444.
[25]
Wonsung Lee, Kyungwoo Song, and Il-Chul Moon. 2017. Augmented variational autoencoders for collaborative filtering with auxiliary information. In CIKM. 1139–1148.
[26]
Yandong Li, Lijun Li, Liqiang Wang, Tong Zhang, and Boqing Gong. 2019. NATTACK: Learning the distributions of adversarial examples for an improved black-box attack on deep neural networks. In ICML. PMLR, 3866–3876.
[27]
Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E. Hopcroft. 2019. Nesterov accelerated gradient and scale invariance for adversarial attacks. arXiv preprint arXiv:1908.06281 (2019).
[28]
Aishan Liu, Xianglong Liu, Hang Yu, Chongzhi Zhang, Qiang Liu, and Dacheng Tao. 2021. Training robust deep neural networks via adversarial noise propagation. Trans. Image Process. 30 (2021), 5769–5781.
[29]
Wanqi Ma, Xiancong Chen, Weike Pan, and Zhong Ming. 2022. VAE++ variational autoencoder for heterogeneous one-class collaborative filtering. In WSDM. 666–674.
[30]
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017).
[31]
Yurii Nesterov. 1983. A method for unconstrained convex minimization problem with the rate of convergence O (1/k̂2303 2). In Doklady USSR. Vol. 269. 543–547.
[32]
Wei Niu, James Caverlee, and Haokai Lu. 2018. Neural personalized ranking for image recommendation. In WSDM. 423–431.
[33]
Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Jun Zhu, and Hang Su. 2020. Boosting adversarial training with hypersphere embedding. Adv. Neural Inf. Process Syst. 33 (2020), 7779–7792.
[34]
Dae Hoon Park and Yi Chang. 2019. Adversarial sampling and training for semi-supervised information retrieval. In WWW. 1443–1453.
[35]
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012).
[36]
Suvash Sedhain, Aditya Krishna Menon, Scott Sanner, and Lexing Xie. 2015. AutoRec: Autoencoders meet collaborative filtering. In WWW. 111–112.
[37]
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S. Davis, Gavin Taylor, and Tom Goldstein. 2019. Adversarial training for free! arXiv preprint arXiv:1904.12843 (2019).
[38]
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013).
[39]
Jinhui Tang, Xiaoyu Du, Xiangnan He, Fajie Yuan, Qi Tian, and Tat-Seng Chua. 2019. Adversarial training towards robust multimedia recommender system. Trans. Knowl. Data Eng. 32, 5 (2019), 855–867.
[40]
Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In WSDM. 565–573.
[41]
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. 2017. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.07204 (2017).
[42]
Jonathan Uesato, Brendan O’Donoghue, Pushmeet Kohli, and Aaron Oord. 2018. Adversarial risk and the dangers of evaluating against weak attacks. In ICML. PMLR, 5025–5034.
[43]
Jiong Wang, Neng Gao, Jia Peng, and Jingjie Mo. 2019. DCAR: Deep collaborative autoencoder for recommendation with implicit feedback. In ICANN. Springer, 172–184.
[44]
Jianfang Wang and Pengfei Han. 2019. Adversarial training-based mean Bayesian personalized ranking for recommender system. IEEE Access 8 (2019), 7958–7968.
[45]
Chenwang Wu, Defu Lian, Yong Ge, Zhihao Zhu, Enhong Chen, and Senchao Yuan. 2021. Fight fire with fire: Towards robust recommender systems via adversarial poisoning training. In SIGIR. ACM, 1074–1083.
[46]
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. 2020. Adversarial weight perturbation helps robust generalization. arXiv preprint arXiv:2004.05884 (2020).
[47]
Yao Wu, Christopher DuBois, Alice X. Zheng, and Martin Ester. 2016. Collaborative denoising auto-encoders for top-n recommender systems. In WISDM. 153–162.
[48]
Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan L. Yuille, and Quoc V. Le. 2020. Adversarial examples improve image recognition. In CVPR. 819–828.
[49]
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. 2017. Mitigating adversarial effects through randomization. arXiv preprint arXiv:1711.01991 (2017).
[50]
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L. Yuille. 2019. Improving transferability of adversarial examples with input diversity. In CVPR. 2730–2739.
[51]
Xin Xin, Bo Chen, Xiangnan He, Dong Wang, Yue Ding, and Joemon Jose. 2019. CFM: Convolutional factorization machines for context-aware recommendation. In IJCAI, Vol. 19. 3926–3932.
[52]
Weilin Xu, David Evans, and Yanjun Qi. 2017. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704.01155 (2017).
[53]
Feng Xue, Xiangnan He, Xiang Wang, Jiandong Xu, Kai Liu, and Richang Hong. 2019. Deep item-based collaborative filtering for top-n recommendation. Trans. Inf. Syst. 37, 3 (2019), 1–25.
[54]
Hong-Jian Xue, Xinyu Dai, Jianbing Zhang, Shujian Huang, and Jiajun Chen. 2017. Deep matrix factorization models for recommender systems. In IJCAI, Vol. 17. 3203–3209.
[55]
Chao Yang, Weixin Zhou, Zhiyu Wang, Bin Jiang, Dongsheng Li, and Huawei Shen. 2021. Accurate and explainable recommendation via hierarchical attention network oriented towards crowd intelligence. Knowl.-based Syst. 213 (2021), 106687.
[56]
Mingyang Yi, Lu Hou, Jiacheng Sun, Lifeng Shang, Xin Jiang, Qun Liu, and Zhi-Ming Ma. 2021. Improved OOD generalization via adversarial training and pre-training. arXiv preprint arXiv:2105.11144 (2021).
[57]
Feng Yuan, Lina Yao, and Boualem Benatallah. 2019. Adversarial collaborative auto-encoder for top-n recommendation. In IJCNN. IEEE, 1–8.
[58]
Feng Yuan, Lina Yao, and Boualem Benatallah. 2019. Adversarial collaborative neural network for robust recommendation. In SIGIR. 1065–1068.
[59]
Quangui Zhang, Longbing Cao, Chengzhang Zhu, Zhiqiang Li, and Jinguang Sun. 2018. CoupleDCF: Learning explicit and implicit user-item couplings in recommendation for deep collaborative filtering. In IJCAI.
[60]
Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep learning based recommender system: A survey and new perspectives. ACM Comput. Surv. 52, 1 (2019), 1–38.
[61]
Yongfeng Zhang, Qingyao Ai, Xu Chen, and W. Bruce Croft. 2017. Joint representation learning for top-n recommendation with heterogeneous information sources. In CIKM. 1449–1458.

Cited By

View all
  • (2024)A Comprehensive Understanding of the Impact of Data Augmentation on the Transferability of 3D Adversarial ExamplesACM Transactions on Knowledge Discovery from Data10.1145/3673232Online publication date: 15-Jun-2024
  • (2024)Fairness and Diversity in Recommender Systems: A SurveyACM Transactions on Intelligent Systems and Technology10.1145/3664928Online publication date: 21-May-2024
  • (2024)ID-SR: Privacy-Preserving Social Recommendation Based on Infinite Divisibility for Trustworthy AIACM Transactions on Knowledge Discovery from Data10.1145/363941218:7(1-25)Online publication date: 19-Jun-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Information Systems
ACM Transactions on Information Systems  Volume 42, Issue 1
January 2024
924 pages
EISSN:1558-2868
DOI:10.1145/3613513
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 August 2023
Accepted: 20 March 2023
Revised: 14 March 2023
Received: 22 May 2022
Published in TOIS Volume 42, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Recommendation systems
  2. deep collaborative filtering
  3. adversarial training

Qualifiers

  • Research-article

Funding Sources

  • General Project of the National Natural Science Foundation of China
  • General Project of the Natural Science Foundation of Anhui Province
  • Scientific Research Planning Project of Anhui Province

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)234
  • Downloads (Last 6 weeks)30
Reflects downloads up to 18 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)A Comprehensive Understanding of the Impact of Data Augmentation on the Transferability of 3D Adversarial ExamplesACM Transactions on Knowledge Discovery from Data10.1145/3673232Online publication date: 15-Jun-2024
  • (2024)Fairness and Diversity in Recommender Systems: A SurveyACM Transactions on Intelligent Systems and Technology10.1145/3664928Online publication date: 21-May-2024
  • (2024)ID-SR: Privacy-Preserving Social Recommendation Based on Infinite Divisibility for Trustworthy AIACM Transactions on Knowledge Discovery from Data10.1145/363941218:7(1-25)Online publication date: 19-Jun-2024

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media