Wenjie Wang
ABSTRACT
Recommender systems rely on user-item interactions to learn Collaborative Filtering (CF) signals and easily under-recommend the cold-start items without historical interactions. To boost cold-start item recommendation, previous studies usually incorporate item features (e.g., micro-video content features) into CF models. They essentially align the feature representations of warm-start items with CF representations during training, and then adopt the feature representations of cold-start items to make recommendations. However, cold-start items might have feature distribution shifts from warm-start ones due to different upload times. As such, these cold-start item features fall into the underrepresented feature space, where their feature representations cannot align well with CF signals, causing poor cold-start recommendation.
To combat item feature shifts, the key lies in pushing feature representation learning to well represent the shifted item features and align with the CF representations in the underrepresented feature space. To this end, we propose an equivariant learning framework, which aims to achieve equivariant alignment between item features, feature representations, and CF representations in the underrepresented feature space. Specifically, since cold-start items are unavailable for training, we interpolate the features and CF representations of two underrepresented warm items to simulate the feature shifts. The interpolated feature representations are then regulated to achieve equivariant alignment with the interpolated features and CF representations via three alignment losses. We instantiate the proposed framework on two competitive cold-start models, and empirical results on three datasets validate that the framework significantly improves cold-start recommendation.
- Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. 2019. Invariant risk minimization. In arXiv:1907.02893.Google Scholar
- Yogesh Balaji, Swami Sankaranarayanan, and Rama Chellappa. 2018. Metareg: Towards domain generalization using meta-regularization. NeurIPS.Google Scholar
- Iman Barjasteh, Rana Forsati, Dennis Ross, Abdol-Hossein Esfahanian, and Hayder Radha. 2016. Cold-start recommendation with provable guarantees: A decoupled approach. TKDE, Vol. 28, 6 (2016), 1462--1474.Google ScholarDigital Library
- Oren Barkan, Noam Koenigstein, Eylon Yogev, and Ori Katz. 2019. CB2CF: a neural multiview content-to-collaborative filtering model for completely cold item recommendations. In RecSys. ACM, 228--236.Google Scholar
- Peter Bühlmann. 2020. Invariance, causality and robustness. Statist. Sci., Vol. 35, 3 (2020), 404--426.Google Scholar
- Tianwei Cao, Qianqian Xu, Zhiyong Yang, and Qingming Huang. 2020. Task-distribution-aware Meta-learning for Cold-start CTR Prediction. In MM. ACM, 3514--3522.Google Scholar
- Hao Chen, Zefan Wang, Feiran Huang, Xiao Huang, Yue Xu, Yishi Lin, Peng He, and Zhoujun Li. 2022a. Generative Adversarial Framework for Cold-Start Item Recommendation. In SIGIR. ACM, 2565--2571.Google Scholar
- Junyu Chen, Qianqian Xu, Zhiyong Yang, Ke Ma, Xiaochun Cao, and Qingming Huang. 2022b. Recurrent Meta-Learning against Generalized Cold-start Problem in CTR Prediction. In MM. ACM, 2636--2644.Google Scholar
- Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, and Marin Soljacic. 2022. Equivariant Self-Supervised Learning: Encouraging Equivariance in Representations. In ICLR.Google Scholar
- Yashar Deldjoo, Maurizio Ferrari Dacrema, Mihai Gabriel Constantin, Hamid Eghbal-Zadeh, Stefano Cereda, Markus Schedl, Bogdan Ionescu, and Paolo Cremonesi. 2019. Movie genome: alleviating new item cold start in movie recommendation. UMUAI, Vol. 29, 2 (2019), 291--343.Google ScholarDigital Library
- Yashar Deldjoo, Markus Schedl, Paolo Cremonesi, and Gabriella Pasi. 2020. Recommender systems leveraging multimedia content. CSUR, Vol. 53, 5 (2020), 1--38.Google ScholarDigital Library
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. textBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In arXiv:1810.04805.Google Scholar
- Jing Du, Zesheng Ye, Lina Yao, Bin Guo, and Zhiwen Yu. 2022b. Socially-aware Dual Contrastive Learning for Cold-Start Recommendation. In SIGIR. ACM, 1927--1932.Google ScholarDigital Library
- Xiaoyu Du, Xiang Wang, Xiangnan He, Zechao Li, Jinhui Tang, and Tat-Seng Chua. 2020. How to learn item representation for cold-start multimedia recommendation?. In MM. ACM, 3469--3477.Google Scholar
- Xiaoyu Du, Zike Wu, Fuli Feng, Xiangnan He, and Jinhui Tang. 2022a. Invariant Representation Learning for Multimedia Recommendation. In MM. ACM.Google Scholar
- Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML. PMLR, 1126--1135.Google Scholar
- Chen Gao, Yu Zheng, Wenjie Wang, Fuli Feng, Xiangnan He, and Yong Li. 2022. Causal inference in recommender systems: A survey and future directions. arXiv:2208.12397.Google Scholar
- Xue Geng, Hanwang Zhang, Jingwen Bian, and Tat-Seng Chua. 2015. Learning image and user features for recommendation in social networks. In ICCV. IEEE, 4274--4282.Google Scholar
- Karan Goel, Albert Gu, Yixuan Li, and Christopher Ré. 2021. Model patching: Closing the subgroup performance gap with data augmentation. In ICLR.Google Scholar
- Cheng Guo, Hongyu Lu, Shaoyun Shi, Bin Hao, Bin Liu, Min Zhang, Yiqun Liu, and Shaoping Ma. 2017. How integration helps on cold-start recommendations. In RecSys Challenge. ACM, 1--6.Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. IEEE, 770--778.Google Scholar
- Ruining He and Julian McAuley. 2016. Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering. In WWW. ACM, 507--517.Google Scholar
- Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural Collaborative Filtering. In WWW. ACM, 173--182.Google Scholar
- Yue He, Zimu Wang, Peng Cui, Hao Zou, Yafeng Zhang, Qiang Cui, and Yong Jiang. 2022. CausPref: Causal Preference Learning for Out-of-Distribution Recommendation. In WWW. ACM, 410--421.Google Scholar
- Zhaoxin Huan, Gongduo Zhang, Xiaolu Zhang, Jun Zhou, Qintong Wu, Lihong Gu, Jinjie Gu, Yong He, Yue Zhu, and Linjian Mo. 2022. An Industrial Framework for Cold-Start Recommendation in Zero-Shot Scenarios. In SIGIR. ACM, 3403--3407.Google Scholar
- Jin Huang, Harrie Oosterhuis, Maarten De Rijke, and Herke Van Hoof. 2020. Keeping dataset biases out of the simulation: A debiased simulator for reinforcement learning based recommender systems. In RecSys. ACM, 190--199.Google Scholar
- Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer, Vol. 42, 8 (2009), 30--37.Google ScholarDigital Library
- Hoyeop Lee, Jinbae Im, Seongwon Jang, Hyunsouk Cho, and Sehee Chung. 2019. Melu: Meta-learned User Preference Estimator for Cold-start Recommendation. In KDD. ACM, 1073--1082.Google ScholarDigital Library
- Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy Hospedales. 2018. Learning to generalize: Meta-learning for domain generalization. In AAAI, Vol. 32. ACM.Google ScholarCross Ref
- Jingjing Li, Mengmeng Jing, Ke Lu, Lei Zhu, Yang Yang, and Zi Huang. 2019. From zero-shot learning to cold-start recommendation. In AAAI, Vol. 33. AAAI press, 4189--4196.Google Scholar
- Jingjing Li, Ke Lu, Zi Huang, and Heng Tao Shen. 2017. Two birds one stone: on both cold-start and long-tail recommendation. In MM. ACM, 898--906.Google Scholar
- Wenhui Li, Yan Wang, Yuting Su, Xuanya Li, An-An Liu, and Yongdong Zhang. 2023. Multi-Scale Fine-Grained Alignments for Image and Sentence Matching. TMM, Vol. 25 (2023), 543--556.Google ScholarCross Ref
- Xixun Lin, Jia Wu, Chuan Zhou, Shirui Pan, Yanan Cao, and Bin Wang. 2021. Task-adaptive Neural Process for User Cold-Start Recommendation. In WWW. ACM, 1306--1316.Google Scholar
- Fan Liu, Huilin Chen, Zhiyong Cheng, Anan Liu, Liqiang Nie, and Mohan Kankanhalli. 2022a. Disentangled Multimodal Representation Learning for Recommendation. TMM (2022), 1--11.Google Scholar
- Huafeng Liu, Liping Jing, Dahai Yu, Mingjie Zhou, and Michael Ng. 2022b. Learning Intrinsic and Extrinsic Intentions for Cold-start Recommendation with Neural Stochastic Processes. In MM. ACM, 491--500.Google Scholar
- Jiashuo Liu, Zheyuan Hu, Peng Cui, Bo Li, and Zheyan Shen. 2021. Heterogeneous risk minimization. In ICML. PMLR, 6804--6814.Google Scholar
- Yuanfu Lu, Yuan Fang, and Chuan Shi. 2020. Meta-learning on heterogeneous information networks for cold-start recommendation. In SIGKDD. ACM, 1563--1573.Google Scholar
- Marco Morik, Ashudeep Singh, Jessica Hong, and Thorsten Joachims. 2020. Controlling Fairness and Bias in Dynamic Learning-to-Rank. In SIGIR. ACM, 429--438.Google Scholar
- Feiyang Pan, Shuokai Li, Xiang Ao, Pingzhong Tang, and Qing He. 2019. Warm up cold-start advertisements: Improving ctr predictions via learning to learn id embeddings. In SIGIR. ACM, 695--704.Google Scholar
- Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. 2020. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. In ICLR.Google Scholar
- Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata. 2020. Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback. In WSDM. ACM, 501--509.Google Scholar
- Jitao Sang, Ming Yan, and Changsheng Xu. 2018. Understanding dynamic cross-OSN associations for cold-start recommendation. TMM, Vol. 20, 12 (2018), 3439--3451.Google ScholarDigital Library
- Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In WWW. ACM, 285--295.Google Scholar
- Martin Saveski and Amin Mantrach. 2014. Item cold-start recommendations: learning local collective embeddings. In ResSys. ACM, 89--96.Google Scholar
- Walid Shalaby, Sejoon Oh, Amir Afsharinejad, Srijan Kumar, and Xiquan Cui. 2022. M2TRec: metadata-aware multi-task transformer for large-scale and cold-start free session-based recommendations. In RecSys. ACM, 573--578.Google Scholar
- Shaoyun Shi, Min Zhang, Yiqun Liu, and Shaoping Ma. 2018. Attention-based adaptive model to unify warm and cold starts recommendation. In CIKM. ACM, 127--136.Google Scholar
- Shaoyun Shi, Min Zhang, Xinxing Yu, Yongfeng Zhang, Bin Hao, Yiqun Liu, and Shaoping Ma. 2019. Adaptive feature sampling for recommendation with missing content feature values. In CIKM. ACM, 1451--1460.Google Scholar
- Xuehan Sun, Tianyao Shi, Xiaofeng Gao, Yanrong Kang, and Guihai Chen. 2021. FORM: follow the online regularized meta-leader for cold-start recommendation. In SIGIR. ACM, 1177--1186.Google Scholar
- Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. 2017. Domain randomization for transferring deep neural networks from simulation to the real world. In IROS. IEEE, 23--30.Google Scholar
- Manasi Vartak, Arvind Thiagarajan, Conrado Miranda, Jeshua Bratman, and Hugo Larochelle. 2017. A meta-learning perspective on cold-start recommendations for items. NeurIPS.Google Scholar
- Maksims Volkovs, Guangwei Yu, and Tomi Poutanen. 2017. Dropoutnet: Addressing cold start in recommender systems. NeurIPS.Google Scholar
- Jianling Wang, Kaize Ding, and James Caverlee. 2021a. Sequential Recommendation for Cold-Start Users with Meta Transitional Learning. In SIGIR. ACM, 1783--1787.Google Scholar
- Shuai Wang, Kun Zhang, Le Wu, Haiping Ma, Richang Hong, and Meng Wang. 2021c. Privileged graph distillation for cold start recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 1187--1196.Google ScholarDigital Library
- Tan Wang, Zhongqi Yue, Jianqiang Huang, Qianru Sun, and Hanwang Zhang. 2021b. Self-supervised learning disentangled group representation as feature. NeurIPS, 18225--18240.Google Scholar
- Wenjie Wang, Xinyu Lin, Fuli Feng, Xiangnan He, Min Lin, and Tat-Seng Chua. 2022. Causal Representation Learning for Out-of-Distribution Recommendation. In WWW. ACM, 3562--3571.Google Scholar
- Wenjie Wang, Yang Zhang, Haoxuan Li, Peng Wu, Fuli Feng, and Xiangnan He. 2023. Causal recommendation: Progresses and future directions. In SIGIR. ACM, 3432--3435.Google Scholar
- Yinwei Wei, Xiang Wang, Qi Li, Liqiang Nie, Yan Li, Xuanping Li, and Tat-Seng Chua. 2021. Contrastive learning for cold-start recommendation. In MM. ACM, 5382--5390.Google Scholar
- Hongyi Wen, Xinyang Yi, Tiansheng Yao, Jiaxi Tang, Lichan Hong, and Ed H Chi. 2022. Distributionally-robust recommendations for improving worst-case user experience. In WWW. ACM, 3606--3610.Google Scholar
- Xiaoxiao Xu, Chen Yang, Qian Yu, Zhiwei Fang, Jiaxing Wang, Chaosheng Fan, Yang He, Changping Peng, Zhangang Lin, and Jingping Shao. 2022. Alleviating Cold-start Problem in CTR Prediction with A Variational Embedding Learning Framework. In WWW. ACM, 27--35.Google Scholar
- Huaxiu Yao, Yu Wang, Sai Li, Linjun Zhang, Weixin Liang, James Zou, and Chelsea Finn. 2022. Improving Out-of-Distribution Robustness via Selective Augmentation. In ICML. PMLR.Google Scholar
- Runsheng Yu, Yu Gong, Xu He, Yu Zhu, Qingwen Liu, Wenwu Ou, and Bo An. 2021. Personalized Adaptive Meta Learning for Cold-start User Preference Prediction. In AAAI. AAAI Press, 10772--10780.Google Scholar
- Fajie Yuan, Xiangnan He, Alexandros Karatzoglou, and Liguang Zhang. 2020. Parameter-efficient Transfer from Sequential Behaviors for User Modeling and Recommendation. In SIGIR. ACM, 1469--1478.Google Scholar
- Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz. 2018. mixup: Beyond Empirical Risk Minimization. In ICLR.Google Scholar
- Xinran Zhang, Xin Yuan, Yunwei Li, and Yanru Zhang. 2019. Cold-Start representation learning: A recommendation approach with bert4Movie and movie2Vec. In MM. ACM, 2612--2616.Google Scholar
- Xu Zhao, Yi Ren, Ying Du, Shenzheng Zhang, and Nian Wang. 2022. Improving Item Cold-start Recommendation via Model-agnostic Conditional Variational Autoencoder. arXiv preprint arXiv:2205.13795 (2022).Google Scholar
- Yongchun Zhu, Kaikai Ge, Fuzhen Zhuang, Ruobing Xie, Dongbo Xi, Xu Zhang, Leyu Lin, and Qing He. 2021a. Transfer-meta framework for cross-domain recommendation to cold-start users. In SIGIR. ACM, 1813--1817.Google ScholarDigital Library
- Yongchun Zhu, Zhenwei Tang, Yudan Liu, Fuzhen Zhuang, Ruobing Xie, Xu Zhang, Leyu Lin, and Qing He. 2022. Personalized transfer of user preferences for cross-domain recommendation. In WSDM. ACM, 1507--1515.Google Scholar
- Yongchun Zhu, Ruobing Xie, Fuzhen Zhuang, Kaikai Ge, Ying Sun, Xu Zhang, Leyu Lin, and Juan Cao. 2021b. Learning to warm up cold item embeddings for cold-start recommendation with meta scaling and shifting networks. In SIGIR. ACM, 1167--1176.Google Scholar
- Ziwei Zhu, Shahin Sefati, Parsa Saadatpanah, and James Caverlee. 2020. Recommendation for new users and new items via randomized training and mixture-of-experts transformation. In SIGIR. ACM, 1121--1130.Google Scholar
Index Terms
- Equivariant Learning for Out-of-Distribution Cold-start Recommendation
Recommendations
Contrastive Learning for Cold-Start Recommendation
MM '21: Proceedings of the 29th ACM International Conference on MultimediaRecommending purely cold-start items is a long-standing and fundamental challenge in the recommender systems. Without any historical interaction on cold-start items, the collaborative filtering (CF) scheme fails to leverage collaborative signals to ...
Generative Adversarial Framework for Cold-Start Item Recommendation
SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information RetrievalThe cold-start problem has been a long-standing issue in recommendation. Embedding-based recommendation models provide recommendations by learning embeddings for each user and item from historical interactions. Therefore, such embedding-based models ...
Contrastive Collaborative Filtering for Cold-Start Item Recommendation
WWW '23: Proceedings of the ACM Web Conference 2023The cold-start problem is a long-standing challenge in recommender systems. As a promising solution, content-based generative models usually project a cold-start item’s content onto a warm-start item embedding to capture collaborative signals from item ...
Comments