Skip to main content
SpringerLink
Log in

Robust multi-objective visual bayesian personalized ranking for multimedia recommendation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Machine learning classifiers are susceptible to adversarial perturbations, and their existence raises security concerns with a focus on recommendation systems. While there is a substantial effort to investigate attacks and defensive techniques in recommendation systems, Basic Iterative perturbation strategies (BIM) have been under-researched in multimedia recommendation. In this work, we adapt the iterative approach for multimedia recommendation. We proposed a novel Dynamic Collaborative Filtering with Aesthetic (DCFA) approach which leverages aesthetic features of clothing images into a multi-objective pairwise ranking to capture consumer aesthetic taste at a specific time through adversarial training (ADCFA). The DCFA method extends visual recommendation to make three key contributions: (1) incorporate aesthetic features into multimedia recommender system to model consumers’ preferences in the aesthetic aspect. (2) Design a multi-objective personalized ranking for the visual recommendation. (3) Use the aesthetic features to optimize the learning strategy to capture the temporal dynamics of image aesthetic preferences. To reduce the impact of perturbation, we train a DCFA objective function using minimax adversarial training. Extensive experiments on three datasets demonstrate the effectiveness of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Wang M, Liu X, Wu X (2015) Visual classification by l1-hypergraph modeling [J]. IEEE Trans on Knowl and Data Eng:2564–2574

  2. He R, McAuley J (2016) VBPR: Visual bayesian personalized ranking from implicit feedback [C]. In: AAAI, pp 144–150

  3. Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L (2009) BPR: Bayesian Personalized ranking from implicit feedback [C]. UAI:452–461

  4. He X, Liao L, Zhang H, Nie L, Hu X, Chua T S (2017) Neural collaborative filtering [C]. In: WWW, pp 173–182

  5. Su J H, Huang W J, Philip S Y, Tseng V S (2011) Efficient relevance feedback for content-based image retrieval by mining user navigation patterns [J]. IEEE Trans Knowl Data Eng:360–372

  6. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition [C]. In: CVPR, pp 770–778

  7. Chen J, Zhang H, He X, Nie L, Liu W, Chua T S (2017) Attentive collaborative filtering: Multimedia recommendation with item and component-level attention [C]. In: SIGIR, pp 335–344

  8. Cheng Z, Chang X, Zhu L, Kanjirathinkal R C, Kankanhalli M (2019) Mmalfm: Explainable recommendation by leveraging reviews and images [J]. In: TOIS, vol 16, pp 1–16, vol 28

  9. Lei C, Liu D, Li W, Zha Z J, Li H (2016) Comparative deep learning of hybrid representations for image recommendations [C]. In: CVPR, pp 2545–2553

  10. Yu W, Zhang H, He X, Chen X, Xiong L, Qin Z (2019) Aesthetic-based clothing recommendation [C]. In: WWW, pp 649–658

  11. Yu W, He X, Pei J, Chen X, Xiong L, Liu J, Qin Z (2019) Visually-aware Recommendation with Aesthetic Features [J]. arXiv:1905.02009

  12. Huang L, Joseph A D, Nelson B, Rubinstein B I P, Tygar J D (2011) Adversarial machine learning [C]. In: ACM, pp 43–58

  13. Akhtar N, Mian A S (2018) Threat of adversarial attacks on deep learning in computer vision: a survey [J]. IEEE Access 6:14410–14430

  14. Zhang W E, Sheng Q Z, Alhazmi A, Li C (2020) Adversarial attacks on Deep-Learning models in natural language processing: a survey [J]. ACM Trans Intell Syst Technol, pp 41

  15. Deldjoo Y, Noia T D, Merra F A (2020) Adversarial Machine Learning in Recommender Systems (AML-recsys) [C]. In: ACM, pp 869–872

  16. Goodfellow I J, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples [C]. In: ICLR. arXiv:1412.6572

  17. Kurakin A, Goodfellow I J, Bengio S (2017) Adversarial examples in the physical world [C]. In: ICLR. https://openreview.net/forum?id=HJGU3Rodl

  18. He X, He Z, Du X, Chua T S (2018) Adversarial personalized ranking for recommendation [C]. In: SIGIR, pp 355– 364

  19. Tang J, Du X, He X, Yuan F, Tian Q, Chua T S (2019) Adversarial training towards robust multimedia recommender system [J]. IEEE Trans on Knowl and Data Eng:1–1

  20. Di Noia T, Malitesta D, Merra F A (2020) TAAMR: Targeted Adversarial Attack against Multimedia Recommender Systems [C]. DSN Worksh:1–8

  21. Ricci F, Rokach L, Shapira B, Kantor P B (2011) Recommender systems handbook —— advances in collaborative filtering. [M]. Springer, pp 145–186

  22. Hu Y, Koren Y, Volinsky C (2008) Collaborative filtering for implicit feedback datasets [C]. In: ICDM, pp 263–272

  23. Goldberg D, Nichols D A, Oki B M, Terry D B (1992) Using collaborative filtering to weave an information tapestry [J]. In: ACM, pp 61–70

  24. Anelli V W, Noia T D, Sciascio E D, Ragone A, Trotta J (2019) How to make latent factors interpretable by feeding factorization machines with knowledge graphs [C]. In: ISWC, pp 38–56

  25. Chen L, Chen G, Wang F (2015) Recommender systems based on user reviews: the state of the art [J]. User Model. User-Adapt Interact, pp 99–154

  26. He R, McAuley J (2016) Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering [C]. In: WWW, pp 507–517

  27. Lops P, Gemmis M D, Semeraro G (2011) Content-based recommender systems: State of the art and trends [M]. Springer, pp 73–105

  28. Wu L, Chen L, Hong R, Fu Y, Xie X, Wang M (2020) A hierarchical attention model for social contextual image recommendation [J]. IEEE Trans Knowl Data Eng 32(10):1854– 1867

  29. Yin R, Li K, Lu J, Zhang G (2019) Enhancing fashion recommendation with visual compatibility relationship [C]. In: WWW, pp 3434–3440

  30. Jian M, Jia T, Yang X, Wu L, Huo L (2019) Cross-modal Collaborative Manifold Propagation for Image Recommendation [C]. In: ICMR, pp 344–348

  31. Kang W C, Fang C, Wang Z, McAuley J J (2017) Visually-Aware Fashion recommendation and design with generative image models [C]. In: ICDM, pp 207–216

  32. Geng X, Zhang H, Bian J, Chua T S (2015) Learning image and user features for recommendation in social networks [C]. In: ICCV, pp 4274–4282

  33. Deldjoo Y, Elahi M, Cremonesi P, Garzotto F, Piazzolla P, Quadrana M (2016) Content-based video recommendation system based on stylistic visual features [J]. J Data Semant:99– 113

  34. Donaldson J (2007) A hybrid social-acoustic recommendation system for popular music [C]. In: ACM, pp 187–190

  35. Wang S, Wang Y, Tang J, Shu K, Ranganath S, Liu H (2017) What Your Images Reveal: Exploiting Visual Contents for Point-of-Interest Recommendation [C]. WWW, pp 391–400

  36. Liu Q, Wu S, Wang L (2017) Deepstyle: Learning User Preferences for Visual Recommendation [C]. In: SIGIR, pp 841–844

  37. Deldjoo Y, Noia T D, Merra F A (2019) Assessing the impact of a user-item collaborative attack on class of users [C]. In: ACM. http://ceur-ws.org/Vol-2462/paper2.pdf

  38. Anelli V W, Deldjoo Y, Noia T D, Sciascio E D, Merra F A (2020) Sasha: Semantic-aware shilling attacks on recommender systems exploiting knowledge graphs [C]. In: ESWC, pp 307–323. https://doi.org/10.1007/978-3-030-49461-2_18

  39. Fang M, Gong N Z, Liu J (2020) Influence function based data poisoning attacks to top-n recommender systems [C]. In: WWW, pp 3019–3025

  40. Fang M, Yang G, Gong N Z, Liu J (2018) Poisoning attacks to graph-based recommender systems [C]. In: ACSAC, pp 381–392

  41. Lam S K, Riedl J (2004) Shilling recommender systems for fun and profit [C]. In: WWW, pp 393–402

  42. O’Mahony M P, Hurley N J, Kushmerick N, Silvestre G C M (2004) Collaborative recommendation: A robustness analysis [J]. ACM Trans Internet Techn:344–377

  43. Chen H, Li J (2019) Adversarial tensor factorization for context-aware recommendation [C]. RecSys:363–367

  44. Yuan F, Yao L, Benatallah B (2019) Adversarial collaborative neural network for robust recommendation [C]. In: SIGIR, pp 1065–1068

  45. Du Y, Fang M, Yi J, Xu C, Cheng J, Tao D (2019) Enhancing the robustness of neural collaborative filtering systems under malicious attacks [J]. IEEE Trans. Multimedia, pp 555–565

  46. Rafailidis D, Crestani F (2019) Adversarial training for review-based recommendations [C]. In: SIGIR, pp 1057–1060

Download references

Author information

Authors and Affiliations

  1. College of Information Engineering, Zhejiang University of Technology, Hangzhou, 310023, People’s Republic of China

    Agyemang Paul, Zhefu Wu, Kai Liu & Shufeng Gong

Authors
  1. Agyemang Paul
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhefu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shufeng Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhefu Wu.

Ethics declarations

Competing interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Paul, A., Wu, Z., Liu, K. et al. Robust multi-objective visual bayesian personalized ranking for multimedia recommendation. Appl Intell 52, 3499–3510 (2022). https://doi.org/10.1007/s10489-021-02355-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02355-w

Keywords

Search

Navigation