research-article

Mutually-Regularized Dual Collaborative Variational Auto-encoder for Recommendation Systems

Authors:

Zhenzhong ChenAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 2379 - 2387

https://doi.org/10.1145/3485447.3512110

Published: 25 April 2022 Publication History

Abstract

Recently, user-oriented auto-encoders (UAEs) have been widely used in recommender systems to learn semantic representations of users based on their historical ratings. However, since latent item variables are not modeled in UAE, it is difficult to utilize the widely available item content information when ratings are sparse. In addition, whenever new items arrive, we need to wait for collecting rating data for these items and retrain the UAE from scratch, which is inefficient in practice. Aiming to address the above two problems simultaneously, we propose a mutually-regularized dual collaborative variational auto-encoder (MD-CVAE) for recommendation. First, by replacing randomly initialized last layer weights of the vanilla UAE with stacked latent item embeddings, MD-CVAE integrates two heterogeneous information sources, i.e., item content and user ratings, into the same principled variational framework where the weights of UAE are regularized by item content such that convergence to a non-optima due to data sparsity can be avoided. In addition, the regularization is mutual in that user ratings can also help the dual item content module learn more recommendation-oriented item content embeddings. Finally, we propose a symmetric inference strategy for MD-CVAE where the first layer weights of the UAE encoder are tied to the latent item embeddings of the UAE decoder. Through this strategy, no retraining is required to recommend newly introduced items. Empirical studies show the effectiveness of MD-CVAE in both normal and cold-start scenarios. Codes are available at https://github.com/yaochenzhu/MD-CVAE.

References

[1]

David M Blei, Alp Kucukelbir, and Jon D McAuliffe. 2017. Variational inference: A review for statisticians. J. Amer. Statist. Assoc. 112, 518 (2017), 859–877.

[2]

David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. the Journal of Machine Learning Research 3 (2003), 993–1022.

[3]

Robin Burke. 2002. Hybrid recommender systems: Survey and experiments. User Modeling and User-adapted Interaction 12, 4 (2002), 331–370.

Digital Library

[4]

Yifan Chen and Maarten de Rijke. 2018. A collective variational autoencoder for top-N recommendation with side information. In Proceedings of the 3rd Workshop on Deep Learning for Recommender Systems. 3–9.

Digital Library

[5]

Maurizio Ferrari Dacrema, Paolo Cremonesi, and Dietmar Jannach. 2019. Are we really making much progress? A worrying analysis of recent neural recommendation approaches. In Proceedings of the 13th ACM Conference on Recommender Systems. 101–109.

Digital Library

[6]

Xin Dong, Lei Yu, Zhonghuo Wu, Yuxia Sun, Lingfeng Yuan, and Fangxi Zhang. 2017. A hybrid collaborative filtering model with deep structure for recommender systems. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31.

[7]

Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: A factorization-machine based neural network for CTR prediction. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 1725–1731.

[8]

F Maxwell Harper and Joseph A Konstan. 2015. The movielens datasets: History and context. ACM Transactions on Interactive Intelligent Systems 5, 4 (2015), 1–19.

Digital Library

[9]

Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the 25th International Conference on World Wide Web. 507–517.

Digital Library

[10]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web. 173–182.

Digital Library

[11]

Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In Proceedings of the Eighth IEEE International Conference on Data Mining. 263–272.

Digital Library

[12]

Michael I Jordan, Zoubin Ghahramani, Tommi S Jaakkola, and Lawrence K Saul. 1999. An introduction to variational methods for graphical models. Machine learning 37, 2 (1999), 183–233.

[13]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. In Proceedings of International Conference on Learning Representations.

[14]

Diederik P. Kingma and Max Welling. 2014. Auto-encoding variational Bayes. In International Conference on Learning Representations.

[15]

Yehuda Koren and Robert Bell. 2015. Advances in collaborative filtering. In Recommender Systems Handbook. Springer, 77–118.

Digital Library

[16]

Xiaopeng Li and James She. 2017. Collaborative variational autoencoder for recommender systems. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 305–314.

Digital Library

[17]

Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. 2018. Variational autoencoders for collaborative filtering. In Proceedings of the 2018 World Wide Web Conference. 689–698.

Digital Library

[18]

Kai Luo, Hojin Yang, Ga Wu, and Scott Sanner. 2020. Deep critiquing for VAE-based recommender systems. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1269–1278.

Digital Library

[19]

Jianxin Ma, Chang Zhou, Peng Cui, Hongxia Yang, and Wenwu Zhu. 2019. Learning disentangled representations for recommendation. In Proceedings of the 33rd International Conference on Neural Information Processing Systems. 5711–5722.

[20]

Andriy Mnih and Russ R Salakhutdinov. 2008. Probabilistic matrix factorization. In Proceedings of Advances in Neural Information Processing Systems. 1257–1264.

[21]

Bo Pang, Min Yang, and Chongjun Wang. 2019. A novel top-N recommendation approach based on conditional variational auto-encoder. In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 357–368.

Digital Library

[22]

Suvash Sedhain, Aditya Krishna Menon, Scott Sanner, and Lexing Xie. 2015. AutoRec: Autoencoders meet collaborative filtering. In Proceedings of the 24th International Conference on World Wide Web. 111–112.

Digital Library

[23]

Ilya Shenbin, Anton Alekseev, Elena Tutubalina, Valentin Malykh, and Sergey I Nikolenko. 2020. RecVAE: A new variational autoencoder for top-N recommendations with implicit feedback. In Proceedings of the 13th International Conference on Web Search and Data Mining. 528–536.

Digital Library

[24]

Florian Strub, Romaric Gaudel, and Jérémie Mary. 2016. Hybrid recommender system based on autoencoders. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. 11–16.

Digital Library

[25]

Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and Léon Bottou. 2010. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion.Journal of Machine Learning Research 11, 12 (2010).

[26]

Chong Wang and David M Blei. 2011. Collaborative topic modeling for recommending scientific articles. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 448–456.

Digital Library

[27]

Hao Wang, Naiyan Wang, and Dit-Yan Yeung. 2015. Collaborative deep learning for recommender systems. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1235–1244.

Digital Library

[28]

Yaxiong Wu, Craig Macdonald, and Iadh Ounis. 2020. A hybrid conditional variational autoencoder model for personalised top-N recommendation. In Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval. 89–96.

Digital Library

[29]

Jiayi Xie, Yaochen Zhu, Zhibin Zhang, Jian Peng, Jing Yi, Yaosi Hu, Hongyi Liu, and Zhenzhong Chen. 2020. A multimodal variational encoder-decoder framework for micro-video popularity prediction. In Proceedings of The Web Conference 2020. 2542–2548.

Digital Library

[30]

Jing Yi, Yaochen Zhu, Jiayi Xie, and Zhenzhong Chen. 2021. Cross-modal variational auto-encoder for content-based micro-video background music recommendation. IEEE Transactions on Multimedia(2021).

[31]

Qiaomin Yi, Ning Yang, and Philip Yu. 2021. Dual adversarial variational embedding for robust recommendation. IEEE Trans. Knowl. Data Eng.(2021).

[32]

Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. 2016. Collaborative knowledge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 353–362.

Digital Library

[33]

Yin Zhang, Ziwei Zhu, Yun He, and James Caverlee. 2020. Content-collaborative disentanglement representation learning for enhanced recommendation. In Proceedings of the 14th ACM Conference on Recommender Systems.43–52.

Digital Library

[34]

Yaochen Zhu, Jing Yi, Jiayi Xie, and Zhenzhong Chen. 2022. Deep causal reasoning for recommendations. arXiv preprint arXiv:2201.02088(2022).

Cited By

Peng YGao CZhang YDan TDu XLuo HLi YMeng X(2025)Denoising Alignment with Large Language Model for RecommendationACM Transactions on Information Systems10.1145/369666243:2(1-35)Online publication date: 24-Jan-2025
https://dl.acm.org/doi/10.1145/3696662
Liang SPan Zliu wYin Jde Rijke M(2024)A Survey on Variational Autoencoders in Recommender SystemsACM Computing Surveys10.1145/366336456:10(1-40)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3663364
Guo LZhu YGao MTao YYu JChen CBaeza-Yates RBonchi F(2024)Consistency and Discrepancy-Based Contrastive Tripartite Graph Learning for RecommendationsProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3672056(944-955)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3672056
Show More Cited By

Index Terms

Mutually-Regularized Dual Collaborative Variational Auto-encoder for Recommendation Systems

Index terms have been assigned to the content through auto-classification.

Recommendations

Collaborative Variational Autoencoder for Recommender Systems
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Modern recommender systems usually employ collaborative filtering with rating information to recommend items to users due to its successful performance. However, because of the drawbacks of collaborative-based methods such as sparsity, cold start, etc., ...
Item recommendation in collaborative tagging systems via heuristic data fusion

Collaborative tagging systems have been popular on the Web. However, information overload results in the increasing need for recommender services from users, and thus item recommendation has been one of the key issues in such systems. In this paper, we ...
New Recommendation Techniques for Multicriteria Rating Systems

Traditional single-rating recommender systems have been successful in a number of personalization applications, but the research area of multicriteria recommender systems has been largely untouched. Taking full advantage of multicriteria ratings in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Tencent

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
497
Total Downloads

Downloads (Last 12 months)85
Downloads (Last 6 weeks)8

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Peng YGao CZhang YDan TDu XLuo HLi YMeng X(2025)Denoising Alignment with Large Language Model for RecommendationACM Transactions on Information Systems10.1145/369666243:2(1-35)Online publication date: 24-Jan-2025
https://dl.acm.org/doi/10.1145/3696662
Liang SPan Zliu wYin Jde Rijke M(2024)A Survey on Variational Autoencoders in Recommender SystemsACM Computing Surveys10.1145/366336456:10(1-40)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3663364
Guo LZhu YGao MTao YYu JChen CBaeza-Yates RBonchi F(2024)Consistency and Discrepancy-Based Contrastive Tripartite Graph Learning for RecommendationsProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3672056(944-955)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3672056
Chen XLei PSheng YLiu YGong ZSerra ESpezzano F(2024)Social Influence Learning for Recommendation SystemsProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679598(312-322)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679598
Zhou XMiao C(2024)Disentangled Graph Variational Auto-Encoder for Multimodal Recommendation With InterpretabilityIEEE Transactions on Multimedia10.1109/TMM.2024.336987526(7543-7554)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3369875
Liu HJing LYu JNg M(2024)Learning Hierarchical Preferences for Recommendation With Mixture Intention Neural Stochastic ProcessesIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.334849336:7(3237-3251)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TKDE.2023.3348493
Liu SLiu JGu HLi DLu TZhang PGu NFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)AutoSeqRec: Autoencoder for Efficient Sequential RecommendationProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614788(1493-1502)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614788
Liu HZhou MJing LNg MEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Doubly Intention Learning for Cold-start Recommendation with Uncertainty-aware Stochastic Meta ProcessProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612446(6212-6222)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612446
Djekidel MManel HKerrache CCalafate C(2023)Toward an Eﬃcient in-Network Caching Using Federated Learning2023 5th International Conference on Pattern Analysis and Intelligent Systems (PAIS)10.1109/PAIS60821.2023.10322015(1-8)Online publication date: 25-Oct-2023
https://doi.org/10.1109/PAIS60821.2023.10322015
Zhu YGeng YLi YQiang JWu X(2023)Representation learning: serial-autoencoder for personalized recommendationFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-023-2441-118:4Online publication date: 16-Dec-2023
https://dl.acm.org/doi/10.1007/s11704-023-2441-1
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten