research-article

Stochastic-Expert Variational Autoencoder for Collaborative Filtering

Authors:

Min-hwan OhAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 2482 - 2490

https://doi.org/10.1145/3485447.3512120

Published: 25 April 2022 Publication History

Abstract

Motivated by the recent successes of deep generative models used for collaborative filtering, we propose a novel framework of VAE for collaborative filtering using multiple experts and stochastic expert selection, which allows the model to learn a richer and more complex latent representation of user preferences. In our method, individual experts are sampled stochastically at each user-item interaction which can effectively utilize the variability among multiple experts. While we propose this framework in the context of collaborative filtering, the proposed stochastic expert technique can be used to enhance VAEs in general beyond the application of collaborative filtering. Hence, this novel technique can be of independent interest. We comprehensively evaluate our proposed method, Stochastic-Expert Variational Autoencoder (SE-VAE) on numerical experiments on the real-world benchmark datasets from MovieLens and Netflix and show that it consistently outperforms the existing state-of-the-art methods across all metrics. Our proposed stochastic expert framework is generic and adaptable to any VAE architecture. The experimental results show that the adaptations to various architectures provided performance gains over the existing methods.

References

[1]

Daniel Billsus, Michael J Pazzani, 1998. Learning collaborative information filters. In Icml, Vol. 98. 46–54.

[2]

Samuel J. Gershman and Noah D. Goodman. 2014. Amortized Inference in Probabilistic Reasoning. Cognitive Science 36(2014).

[3]

Prem Gopalan, Jake M Hofman, and David M Blei. 2015. Scalable Recommendation with Hierarchical Poisson Factorization. In UAI. 326–335.

[4]

Irina Higgins, Loïc Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Lerchner. 2017. beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. In ICLR.

[5]

Matthew D. Hoffman, David M. Blei, Chong Wang, and John Paisley. 2013. Stochastic Variational Inference. Journal of Machine Learning Research 14, 4 (2013), 1303–1347. http://jmlr.org/papers/v14/hoffman13a.html

Digital Library

[6]

Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In 2008 Eighth IEEE International Conference on Data Mining. Ieee, 263–272.

Digital Library

[7]

Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative Filtering for Implicit Feedback Datasets. In 2008 Eighth IEEE International Conference on Data Mining. 263–272. https://doi.org/10.1109/ICDM.2008.22

Digital Library

[8]

Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categorical Reparametrization with Gumbel-Softmax. In Proceedings International Conference on Learning Representations 2017. OpenReviews.net. https://openreview.net/pdf?id=rkE3y85ee

[9]

Daeryong Kim and Bongwon Suh. 2019. Enhancing VAEs for Collaborative Filtering: Flexible Priors & Gating Mechanisms. In Proceedings of the 13th ACM Conference on Recommender Systems (Copenhagen, Denmark) (RecSys ’19). Association for Computing Machinery, New York, NY, USA, 403–407. https://doi.org/10.1145/3298689.3347015

Digital Library

[10]

Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1312.6114

[11]

John Lafferty and David Blei. 2006. Correlated Topic Models. In Advances in Neural Information Processing Systems, Y. Weiss, B. Schölkopf, and J. Platt (Eds.). Vol. 18. MIT Press. https://proceedings.neurips.cc/paper/2005/file/9e82757e9a1c12cb710ad680db11f6f1-Paper.pdf

[12]

Daniel D Lee and H Sebastian Seung. 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401, 6755 (1999), 788–791.

[13]

Dawen Liang, Jaan Altosaar, Laurent Charlin, and David M Blei. 2016. Factorization meets the item embedding: Regularizing matrix factorization with item co-occurrence. In Proceedings of the 10th ACM conference on recommender systems. 59–66.

Digital Library

[14]

Dawen Liang, Rahul G. Krishnan, Matthew D. Hoffman, and Tony Jebara. 2018. Variational Autoencoders for Collaborative Filtering. In Proceedings of the 2018 World Wide Web Conference (Lyon, France) (WWW ’18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 689–698. https://doi.org/10.1145/3178876.3186150

Digital Library

[15]

Sam Lobel, Chunyuan Li, Jianfeng Gao, and Lawrence Carin. 2020. RaCT: Toward Amortized Ranking-Critical Training For Collaborative Filtering. In International Conference on Learning Representations.

[16]

Andriy Mnih and Russ R Salakhutdinov. 2008. Probabilistic matrix factorization. In Advances in neural information processing systems. 1257–1264.

[17]

Xia Ning and George Karypis. 2011. SLIM: Sparse Linear Methods for Top-N Recommender Systems. In Proceedings of the 2011 IEEE 11th International Conference on Data Mining(ICDM ’11). IEEE Computer Society, USA, 497–506. https://doi.org/10.1109/ICDM.2011.134

Digital Library

[18]

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning. PMLR, 1278–1286.

[19]

Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on World Wide Web. 285–295.

Digital Library

[20]

Ilya Shenbin, Anton Alekseev, Elena Tutubalina, Valentin Malykh, and Sergey I. Nikolenko. 2020. RecVAE: A New Variational Autoencoder for Top-N Recommendations with Implicit Feedback. In Proceedings of the 13th International Conference on Web Search and Data Mining (Houston, TX, USA) (WSDM ’20). Association for Computing Machinery, New York, NY, USA, 528–536. https://doi.org/10.1145/3336191.3371831

Digital Library

[21]

Harald Steck. 2019. Embarrassingly shallow autoencoders for sparse data. In The World Wide Web Conference. 3251–3257.

Digital Library

[22]

Xiaoyuan Su and Taghi M Khoshgoftaar. 2009. A survey of collaborative filtering techniques. Advances in artificial intelligence 2009 (2009).

[23]

Jakub Tomczak and Max Welling. 2018. VAE with a VampPrior. In International Conference on Artificial Intelligence and Statistics. PMLR, 1214–1223.

[24]

Yao Wu, Christopher DuBois, Alice X. Zheng, and Martin Ester. 2016. Collaborative Denoising Auto-Encoders for Top-N Recommender Systems. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining (San Francisco, California, USA) (WSDM ’16). Association for Computing Machinery, New York, NY, USA, 153–162. https://doi.org/10.1145/2835776.2835837

Digital Library

[25]

Xianwen Yu, Xiaoning Zhang, Yang Cao, and Min Xia. 2019. VAEGAN: A Collaborative Filtering Framework based on Adversarial Variational Autoencoders. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 4206–4212. https://doi.org/10.24963/ijcai.2019/584

Cited By

Yi JChen Z(2024)Variational Mixture of Stochastic Experts Auto-Encoder for Multi-Modal RecommendationIEEE Transactions on Multimedia10.1109/TMM.2024.338405826(8941-8954)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3384058
Ayemowa MIbrahim RKhan M(2024)Analysis of Recommender System Using Generative Artificial Intelligence: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2024.341696212(87742-87766)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3416962
Chen RFan JWu M(2024)ABNSExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123868250:COnline publication date: 15-Sep-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123868
Show More Cited By

Index Terms

Stochastic-Expert Variational Autoencoder for Collaborative Filtering
1. Computing methodologies
  1. Machine learning
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals

Index terms have been assigned to the content through auto-classification.

Recommendations

Collaborative Variational Autoencoder for Recommender Systems
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Modern recommender systems usually employ collaborative filtering with rating information to recommend items to users due to its successful performance. However, because of the drawbacks of collaborative-based methods such as sparsity, cold start, etc., ...
Trust-based collaborative filtering: tackling the cold start problem using regular equivalence
RecSys '18: Proceedings of the 12th ACM Conference on Recommender Systems

User-based Collaborative Filtering (CF) is one of the most popular approaches to create recommender systems. This approach is based on finding the most relevant k users from whose rating history we can extract items to recommend. CF, however, suffers ...
Bilateral Variational Autoencoder for Collaborative Filtering
WSDM '21: Proceedings of the 14th ACM International Conference on Web Search and Data Mining

Preference data is a form of dyadic data, with measurements associated with pairs of elements arising from two discrete sets of objects. These are users and items, as well as their interactions, e.g., ratings. We are interested in learning ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Institute of Information & communications Technology Planning & Evaluation (IITP)
National Research Foundation of Korea

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
448
Total Downloads

Downloads (Last 12 months)55
Downloads (Last 6 weeks)2

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yi JChen Z(2024)Variational Mixture of Stochastic Experts Auto-Encoder for Multi-Modal RecommendationIEEE Transactions on Multimedia10.1109/TMM.2024.338405826(8941-8954)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3384058
Ayemowa MIbrahim RKhan M(2024)Analysis of Recommender System Using Generative Artificial Intelligence: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2024.341696212(87742-87766)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3416962
Chen RFan JWu M(2024)ABNSExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123868250:COnline publication date: 15-Sep-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123868
Rao QLiu YPan WMing Z(2023)BVAE: Behavior-aware Variational Autoencoder for Multi-Behavior Multi-Task RecommendationProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608781(625-636)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608781
Feng JLiu MLiang XNie T(2023)ALCRKnowledge-Based Systems10.1016/j.knosys.2023.110829278:COnline publication date: 25-Oct-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110829
Feng JLiu MHong SSong S(2023)A Novel Variational Autoencoder with Multi-position Latent Self-attention and Actor-Critic for RecommendationAdvanced Data Mining and Applications10.1007/978-3-031-46661-8_11(155-167)Online publication date: 27-Aug-2023
https://dl.acm.org/doi/10.1007/978-3-031-46661-8_11

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten