Skip to main content
SpringerLink
Log in

Multiple feedback based adversarial collaborative filtering with aesthetics

  • Regular Paper
  • Published:
International Journal of Multimedia Information Retrieval Aims and scope Submit manuscript

Abstract

Visual-aware personalized recommendation systems can estimate the potential demand by evaluating consumer personalized preferences. In general, consumer feedback data is deduced from either explicit feedback or implicit feedback. However, explicit and implicit feedback raises the chance of malicious operation or misoperation, which can lead to deviations in recommended outcomes. Adversarial learning, a regularization approach that can resist disturbances, could be a promising choice for enhancing model resilience. We propose a novel adversarial collaborative filtering with aesthetics (ACFA) for the visual recommendation that utilizes adversarial learning to improve resilience and performance in the case of perturbation. The ACFA algorithm applies three types of input to the visual Bayesian personalized ranking: negative, unobserved, and positive feedback. Through feedbacks at various levels, it uses a probabilistic approach to obtain consumer personalized preferences. Since in visual recommendation, the aesthetic data in determining consumer preferences on product is critical, we construct the consumer personalized preferences model with aesthetic elements, and then use them to enhance the sampling quality when training the algorithm. To mitigate the negative effects of feedback noise, We use minimax adversarial learning to learn the ACFA objective function. Experiments using two datasets demonstrate that the ACFA model outperforms state-of-the-art algorithms on two metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Availability of data and material

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Hu L, Cao J, Xu G et al (2013) Personalized recommendation via cross domain triadic factorization. In: Proceedings of international conference on world wide web, pp 595–606

  2. Deshpande M, Karypis G (2004) Item-based top-N recommendation algorithms. ACM Trans Inf Syst 22(1):143–177

    Article  Google Scholar 

  3. Koren Y (2008) Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: Proceedings of ACM SIGKDD international conference on knowledge discovery and data mining, pp 426–434

  4. Rendle S, Freudenthaler C, Gantner Z et al (2009) BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of uncertainty in artificial intelligence, pp 452–461

  5. Pan R, Zhou Y, Cao B et al (2008) One-class collaborative filtering. In: Proceedings of IEEE international conference on data mining, pp 502–511

  6. Cao Z (2007) Learning to rank: from pairwise approach to listwise approach. In: Proceedings of machine learning, pp 129–136

  7. He R, McAuley J (2016) VBPR: visual bayesian personalized ranking from implicit feedback. In: Proceedings of the thirtieth AAAI conference on artificial intelligence, pp 144–150

  8. Yu W, Zhang H, He X et al (2018) Aesthetic-based clothing recommendation. In: Proceedings of the 2018 world wide web conference on world wide web, pp 649–658

  9. Tang J, Du X, He X, Yuan F, Tian Q, Chua TS (2020) Adversarial training towards robust multimedia recommender system. IEEE Trans Knowl Data Eng 32(5):855–867

    Article  Google Scholar 

  10. Yu DJ, Chen C, Wu JH et al (2018) Personalized game recommendation based on implicit feedback. Chin J Electron 27(4):2626–2632

    Google Scholar 

  11. Liu TY (2009) Learning to rank for information retrieval. Found Trends Inf Ret 3(3):225–331

    Article  Google Scholar 

  12. Zhao T, McAuley J, King I (2014) Leveraging social connections to improve personalized ranking for collaborative filtering. In: Proceedings of ACM international conference on information and knowledge management, pp 261–270

  13. Peugh JL, Enders CK (2004) Missing data in educational research: a review of reporting practices and suggestions for improvement. Rev Educ Res 74(4):525–556

    Article  Google Scholar 

  14. He X, Zhang H, Kan M Y et al (2016) Fast matrix factorization for online recommendation with implicit feedback. In: Proceedings of the 39th international ACM SIGIR conference on research and development in information retrieval, pp 549–558

  15. Runlong Y, Yunzhou Z, Yuyang Y et al (2018) Multiple pairwise ranking with implicit feedback. In: Proceedings of ACM international conference on information and knowledge management, pp 1727–1730

  16. Loftin R, Peng B, MacGlashan J et al (2016) Learning behaviors via human-delivered discrete feedback: modeling implicit feedback strategies to speed up learning. Auton Agent Multi-Ag 30(1):30–59

    Article  Google Scholar 

  17. Chen X, Wang P F, Qin Z et al (2016) Hlbpr: a hybrid local Bayesian personal ranking method. In: Proceedings of international conference on world wide web, pp 21–22

  18. Babak L, Roberto P, Martha L et al (2016) Bayesian personalized ranking with multi-channel user feedback. In: Proceedings of the 10th ACM conference on recommender systems, pp 361–364

  19. He R, Lin C, Wang J, Mcauley J (2016) Sherlock: sparse hierarchical embeddings for visually-aware one-class collaborative filtering. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence, pp 3740–3746

  20. He R, McAuley J (2016) Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering. In: Proceedings of the 25th international conference on world wide web, pp 507–517

  21. Wu Z, Agyemang P, Chan M et al. (2017) Improved one-class collaborative filtering for online recommendation. In: International workshop on complex systems and networks (IWCSN), pp 205–209

  22. Niu W, Caverlee J, Lu H (2018) Neural personalized ranking for image recommendation. In: Proceedings of the eleventh ACM international conference on web search and data mining, pp 423–431

  23. Kang W C, Fang C, Wang Z, McAuley JJ (2017) Visually-aware fashion recommendation and design with generative image models. In: IEEE international conference on data mining, pp 207–216

  24. Liu Q, Wu S, Wang L (2017) DeepStyle: learning user preferences for visual recommendation. In: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, pp 841–844

  25. Meng L, Feng F, He X et al. (2020) Heterogeneous fusion of semantic and collaborative information for visually-aware food recommendation. In: ACM multimedia, pp 3460–3468

  26. Chen J, Zhang H, He X et al (2017) Attentive Collaborative Filtering: Multimedia Recommendation with Item- and Component-Level Attention. In : Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 335-344

  27. Yin R, Li K, Lu J, Zhang G (2019) Enhancing fashion recommendation with visual compatibility relationship. In: The world wide web conference, pp 3434–3440

  28. Paul A, Wu Z, Liu K et al (2022) Robust multi-objective visual bayesian personalized ranking for multimedia recommendation. Appl Intell 52:3499–3510. https://doi.org/10.1007/s10489-021-02355-w

  29. Anelli VW, Deldjoo Y, Noia TD et al. (2021) A study of defensive methods to protect visual recommendation against adversarial manipulation of images. In: SIGIR, pp 1094–1103

  30. Noia TD, Malitesta D, Merra FA (2020) TAaMR: targeted adversarial attack against multimedia recommender systems. In: 50th annual IEEE/IFIP international conference on dependable systems and networks workshops, pp 1–8

  31. Cohen R, Shalom OS, Jannach D, Amir A (2021) A black-box attack model for visually-aware recommender systems. In: The fourteenth ACM international conference on web search and data mining, pp 94–102

  32. Liu Z, Larson M (2021) Adversarial item promotion: vulnerabilities at the core of top-N recommenders that use images to address cold start. In: WWW.ACM/IW3C2, pp 3590–3602

Download references

Acknowledgements

This research was supported by Zhejiang Provincial Natural Science Foundation of China under Grant No. LZ22F010005.

Author information

Authors and Affiliations

  1. College of Information Engineering, Zhejiang University of Technology, Hangzhou, 310023, People’s Republic of China

    Zhefu Wu, Yuhang Ma, Junzhuo Cao & Agyemang Paul

  2. China Comservice HuiZhan Company, Hangzhou, 311215, People’s Republic of China

    Xiang Li

Authors
  1. Zhefu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuhang Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junzhuo Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Agyemang Paul
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiang Li.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, Z., Ma, Y., Cao, J. et al. Multiple feedback based adversarial collaborative filtering with aesthetics. Int J Multimed Info Retr 12, 9 (2023). https://doi.org/10.1007/s13735-023-00273-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13735-023-00273-w

Keywords

Search

Navigation