Skip to main content
SpringerLink
Log in

Robust multimedia recommender system based on dynamic collaborative filtering and directed adversarial learning

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Multimedia Recommendation Systems (MRSs) have shown to be quite effective in learning consumer preferences and recommending the best multimedia products. Recent breakthroughs in adversarial machine learning have piqued the interest of researchers in the security of MRSs. It has been established that widely deployed MRSs are not resilient to detrimental perturbations applied to the learnt parameters, which can result in a significant loss in recommendation accuracy. Adversarial Multimedia Ranking (AMR) mitigates this problem by boosting Visual Bayesian Pairwise Ranking (VBPR) through adversarial learning. The quantitative gains in AMR’s performance on VBPR have led to its widespread implementation in numerous MRS models. However, in MRSs, this strategy overlooks the collaborative feature and is unable to effectively capture the smoothness of data distribution. We contend that modeling MRSs requires the collaborative feature, which displays the behavioral similarity between consumers and products. In this paper, we implement directed adversarial learning with the explicit introduction of the collaborative feature into the perturbation process. Technically, we propose the Adversarial Dynamic Collaborative Filtering (ADCF) for recommendation, which models visual characteristics and captures visual time dynamics. To reduce the influence of perturbation, we train the ADCF objective through minimax adversarial learning. Furthermore, we enhance the ADCF by directed adversarial learning. The objective is to restrict the direction of perturbation in the embedding space to other examples in the present embedding space. This enables us to integrate the collaborative feature into the learning process. Comprehensive evaluations using three Amazon datasets revealed that our technique outperformed baselines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Availability of data and material

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Gorgoglione M, Panniello U, Tuzhilin A (2011) The effect of contextaware recommendations on customer purchasing behavior and trust. In: Proceeding 5th ACM Conference Recommender System, pp 85–92

  2. Krohn-Grimberghe A, Drumond L, Freudenthaler C, Schmidt-Thieme L (2012) Multi-relational matrix factorization using Bayesian personalized ranking for social network data. In: Proceeding 5th ACM International Conference Web Search Data Mining, pp 173–182

  3. Deldjoo Y, Schedl M, Cremonesi P, Pasi G (2020) Recommender systems leveraging multimedia content. ACM Comput Surv 53(5):1061–10638

    Google Scholar 

  4. He R, Mcauley J (2016) VBPR: Visual Bayesian Personalized Ranking from implicit feedback. In: Proceedings of the Thirtieth Conference on Artificial Intelligence, pp 144–150

  5. Yu W, Zhang H, He X, Chen X, Xiong L, Qin Z (2018) Aesthetic-based clothing recommendation. In: Proceedings of the World Wide Web Conference, pp 649–658

  6. Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L (2009) BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pp 452–461

  7. Liu Q, Wu S, and Wang L (2017) Deepstyle: Learning user preferences for visual recommendation. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 841–844

  8. Niu W, Caverlee J, and Lu H (2018) Neural personalized ranking for image recommendation. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pp 423–431

  9. Chen J, Zhang H, He X, Nie L, Liu W, Chua TS (2017) Attentive Collaborative Filtering: Multimedia Recommendation with Item- and Component-Level Attention. In: SIGIR, pp 335–344

  10. Xu C, Guan Z, Zhao W et al (2020) Recommendation by users’ multimodal preferences for smart city applications. IEEE Trans Ind Inf 17(6):4197–4205

    Article  Google Scholar 

  11. Deldjoo Y, Noia TD, Felice Antonio Merra FA (2021) A survey on adversarial recommender systems: from attack/defense strategies to generative adversarial networks. ACM Comput Surv 54(2):1–38

    Article  Google Scholar 

  12. Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples. http://arxiv.org/abs/1412.6572

  13. Madry A, Makelov A, Schmidt L, Tsipras D, and Vladu A (2018) Towards deep learning models resistant to adversarial attacks. http://arxiv.org/abs/1706.06083

  14. Kurakin A, Goodfellow IJ, Bengio S (2017) Adversarial examples in the physical world. http://arxiv.org/abs/1607.02533

  15. Carlini N, Wagner DA (2017) Towards evaluating the robustness of neural networks. In: IEEE Symposium on Security and Privacy, pp 39–57

  16. Miyato T, Maeda SI, Koyama M, Ishii S (2019) Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans Pattern Anal Mach Intell 41(8):1979–1993

    Article  Google Scholar 

  17. Shafahi A, Najibi M, AmGhiasi, Xu Z et al (2019) Adversarial training for free!. In NeurIPS, pp 3353–3364

  18. Szegedy C, Zaremba W, Sutskever I et al (2014) Intriguing properties of neural networks. http://arxiv.org/abs/1312.6199

  19. Krishnan A, Sharma A, Sankar A et al (2018) An adversarial approach to improve long-tail performance in neural collaborative filtering. In: Proceedings of the 27th ACM International Conference on information and knowledge management, pp 1491–1494

  20. He X, He Z, Du X, Chua TS (2018) Adversarial personalized ranking for recommendation. In: The 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 355–364

  21. Tang J, Du X, He X, Yuan F, Tian Q et al (2020) Adversarial training towards robust multimedia recommender system. IEEE Trans Knowl Data Eng 32(5):855–867

    Article  Google Scholar 

  22. Agyemang P, Wu Z, Liu K, Gong S (2022) Robust multi-objective visual Bayesian personalized ranking for multimedia recommendation. Appl Intell 52(4):3499–3510

    Article  Google Scholar 

  23. Agyemang P, Wu Z, Liu K, Gong S (2022) Personalized recommendation: from clothing to academic. Multimed Tools Appl 81(10):14573–14588

    Article  Google Scholar 

  24. Wu Z, Agyemang P, Cao J, Fang L (2022) Directional adversarial training for robust ownership-based recommendation system. IEEE Access 10:2880–2894

    Article  Google Scholar 

  25. Noia TD, Malitesta D, Merra FA (2020) TAaMR: targeted adversarial attack against multimedia recommender systems. In DSN Workshops, pp 1-8

  26. Yuan F, Yao L, Benatallah B (2019) Adversarial collaborative neural network for robust recommendation. In: Proceedings of the 42nd International Conference on Research and Development in Information Retrieval, pp 1065–1068

  27. Chen H, Li J. (2019) Adversarial tensor factorization for contextaware recommendation. In: Proceedings of the 13th Conference on Recommender Systems, pp 363–367

  28. Manotumruksa J, Yilmaz E (2020) Sequential-based adversarial optimisation for personalised top-N Item Recommendation. In: Proceedings of the 43rd International conference on research and development in Information Retrieval, pp 2045–2048

  29. Chong X, Li Q, Leung H, Men Q, Xianjin Chao X (2020) Hierarchical Visual-aware Minimax Ranking Based on Co-purchase Data for Personalized Recommendation. In: WWW, pp 2563–2569

  30. He R, Mcauley J (2016) Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering. In: Proceedings of the 25th International Conference on World Wide Web, pp 507-517

  31. Grauman K (2020) Computer vision for fashion: from individual recommendations to world-wide trends. In: The Thirteenth ACM International Conference on Web Search and Data Mining, p 3

  32. Kang WC, Fang C, Wang Z, McAuley JJ (2017) Visually-aware fashion recommendation and design with generative image models. In: IEEE International Conference on Data Mining, pp 207–216

  33. Yin R, Li K, Lu J, Zhang G (2019) Enhancing fashion recommendation with visual compatibility relationship. In: WWW, pp 3434-3440

  34. Agyemang P, Zhao X, Fang L, Wu Z (2022) Ownership recommendation via iterative adversarial training. Neural Process Lett 54(1):637–655

    Article  Google Scholar 

  35. Yuan F, Yao L, Benatallah B (2020) Exploring missing interactions: a convolutional generative adversarial network for collaborative filtering. In: Proceedings of the 29th ACM International Conference on Information andnowledge Management, pp 1773-1782

Download references

Acknowledgements

This research was supported by Zhejiang Provincial Natural Science Foundation of China under Grant No. LZ22F010005.

Author information

Authors and Affiliations

  1. College of Information Engineering, Zhejiang University of Technology, Hangzhou, 310023, People’s Republic of China

    Agyemang Paul, Zhefu Wu, Kai Luo, Yuhang Ma & Luping Fang

Authors
  1. Agyemang Paul
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhefu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kai Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuhang Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Luping Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhefu Wu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Paul, A., Wu, Z., Luo, K. et al. Robust multimedia recommender system based on dynamic collaborative filtering and directed adversarial learning. Int. J. Mach. Learn. & Cyber. 14, 3851–3865 (2023). https://doi.org/10.1007/s13042-023-01868-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01868-9

Keywords

Search

Navigation