research-article

Aligning Distillation For Cold-start Item Recommendation

Authors:

Hao ChenAuthors Info & Claims

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 1147 - 1157

https://doi.org/10.1145/3539618.3591732

Published: 18 July 2023 Publication History

Abstract

Recommending cold items in recommendation systems is a longstanding challenge due to the inherent differences between warm items, which are recommended based on user behavior, and cold items, which are recommended based on content features. To tackle this, generative models generate synthetic embeddings from content features, while dropout models enhance the robustness of the recommendation system by randomly dropping behavioral embeddings during training. However, these models primarily focus on handling the recommendation of cold items, but do not effectively address the differences between warm and cold recommendations. As a result, generative models may over-recommend either warm or cold items, neglecting the other type, and dropout models may negatively impact warm item recommendations. To address this, we propose the Aligning Distillation (ALDI) framework, which leverages warm items as "teachers" to transfer their behavioral information to cold items, referred to as "students". ALDI aligns the students with the teachers by comparing the differences in their recommendation characters, using tailored rating distribution aligning, ranking aligning, and identification aligning losses to narrow these differences. Furthermore, ALDI incorporates a teacher-qualifying weighting structure to prevent students from learning inaccurate information from unreliable teachers. Experiments on three datasets show that our approach outperforms state-of-the-art baselines in terms of overall, warm, and cold recommendation performance with three different recommendation backbones.

References

[1]

Hessam Bagherinezhad, Maxwell Horton, Mohammad Rastegari, and Ali Farhadi. 2018. Label refinery: Improving imagenet classification through label progression. arXiv preprint arXiv:1805.02641 (2018).

[2]

Dong-Kyu Chae, Jin-Soo Kang, Sang-Wook Kim, and Jaeho Choi. 2019. Rating Augmentation with Generative Adversarial Networks towards Accurate Collaborative Filtering. In WWW.

[3]

Hao Chen, Wenbing Huang, Yue Xu, Fuchun Sun, and Zhoujun Li. 2020a. Graph unfolding networks. In CIKM.

[4]

Hao Chen, Zhong Huang, Yue Xu, Zengde Deng, Feiran Huang, Peng He, and Zhoujun Li. 2022a. Neighbor enhanced graph convolutional networks for node classification and recommendation. Knowledge-Based Systems, Vol. 246 (2022), 108594.

Digital Library

[5]

Hao Chen, Zefan Wang, Feiran Huang, Xiao Huang, Yue Xu, Yishi Lin, Peng He, and Zhoujun Li. 2022b. Generative Adversarial Framework for Cold-Start Item Recommendation. In SIGIR.

[6]

Hao Chen, Yue Xu, Feiran Huang, Zengde Deng, Wenbing Huang, Senzhang Wang, Peng He, and Zhoujun Li. 2020b. Label-aware graph convolutional networks. In CIKM.

[7]

Xu Chen, Yongfeng Zhang, Hongteng Xu, Zheng Qin, and Hongyuan Zha. 2018. Adversarial Distillation for Efficient Recommendation with External Knowledge. TOIS, Vol. 37, 1, Article 12 (dec 2018).

[8]

Xiaoyu Du, Xiang Wang, Xiangnan He, Zechao Li, Jinhui Tang, and Tat-Seng Chua. 2020. How to Learn Item Representation for Cold-Start Multimedia Recommendation?. In MM.

[9]

Shaohua Fan, Junxiong Zhu, Xiaotian Han, Chuan Shi, Linmei Hu, Biyu Ma, and Yongliang Li. 2019. Metapath-Guided Heterogeneous Graph Neural Network for Intent Recommendation. In KDD.

[10]

Wenjing Fu, Zhaohui Peng, Senzhang Wang, Yang Xu, and Jin Li. 2019. Deeply Fusing Reviews and Contents for Cold Start Users in Cross-Domain Recommendation Systems. In AAAI.

[11]

Ishaan Gulrajani, Faruk Ahmed, Martín Arjovsky, Vincent Dumoulin, and Aaron C. Courville. 2017. Improved Training of Wasserstein GANs. In NeurIPS.

Digital Library

[12]

Md. Akmal Haidar and Mehdi Rezagholizadeh. 2019. TextKD-GAN: Text Generation Using Knowledge Distillation and Generative Adversarial Networks. In ICAAI.

[13]

James Hale. 2019. More than 500 hours of content are now being uploaded to YouTube every minute. https://www.tubefilter.com/2019/05/07/number-hours-video-uploaded-to-youtube-per-minute/

[14]

Tong He, Chunhua Shen, Zhi Tian, Dong Gong, Changming Sun, and Youliang Yan. 2019. Knowledge Adaptation for Efficient Semantic Segmentation. In CVPR.

[15]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, YongDong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. In SIGIR.

[16]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural Collaborative Filtering. In WWW.

[17]

John R Hershey and Peder A Olsen. 2007. Approximating the Kullback Leibler divergence between Gaussian mixture models. In ICASSP.

[18]

Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).

[19]

Minghao Hu, Yuxing Peng, Furu Wei, Zhen Huang, Dongsheng Li, Nan Yang, and Ming Zhou. 2018. Attention-Guided Answer Distillation for Machine Reading Comprehension. In EMNLP.

[20]

Zhongyu Huang, Yingheng Wang, Chaozhuo Li, and Huiguang He. 2022. Going Deeper into Permutation-Sensitive Graph Neural Networks. In International Conference on Machine Learning. PMLR.

[21]

Qing-Yuan Jiang, Yi He, Gen Li, Jian Lin, Lei Li, and Wu-Jun Li. 2019. SVD: A Large-Scale Short Video Dataset for Near-Duplicate Video Retrieval. In ICCV.

[22]

SeongKu Kang, Junyoung Hwang, Wonbin Kweon, and Hwanjo Yu. 2020. DE-RRD: A knowledge distillation framework for recommender system. In CIKM.

[23]

Hanyang Kong, Jian Zhao, Xiaoguang Tu, Junliang Xing, Shengmei Shen, and Jiashi Feng. 2019. Cross-resolution face recognition via prior-aided face hallucination and residual knowledge distillation. arXiv preprint arXiv:1905.10777 (2019).

[24]

Yehuda Koren, Robert M. Bell, and Chris Volinsky. 2009. Matrix Factorization Techniques for Recommender Systems. Computer, Vol. 42, 8 (2009), 30--37.

Digital Library

[25]

Jae-woong Lee, Minjin Choi, Jongwuk Lee, and Hyunjung Shim. 2019. Collaborative distillation for top-N recommendation. In ICDM.

[26]

Quanquan Li, Shengying Jin, and Junjie Yan. 2017. Mimicking Very Efficient Network for Object Detection. In CVPR.

[27]

Zhizhong Li and Derek Hoiem. 2018. Learning without Forgetting. TPAMI, Vol. 40, 12 (2018), 2935--2947.

Digital Library

[28]

Hank Liao, Erik McDermott, and Andrew Senior. 2013. Large scale deep neural network acoustic modeling with semi-supervised training data for YouTube video transcription. In ASRU.

[29]

Jian Liu, Yubo Chen, and Kang Liu. 2019. Exploiting the Ground-Truth: An Adversarial Imitation Based Knowledge Distillation Approach for Event Detection. In AAAI.

[30]

Zhicong Lu, Haijun Xia, Seongkook Heo, and Daniel Wigdor. 2018. You watch, you give, and you engage: a study of live streaming practices in China. In CHI.

[31]

Ping Luo, Zhenyao Zhu, Ziwei Liu, Xiaogang Wang, and Xiaoou Tang. 2016. Face model compression by distilling knowledge from neurons. In AAAI.

[32]

Ravi Teja Mullapudi, Steven Chen, Keyi Zhang, Deva Ramanan, and Kayvon Fatahalian. 2019. Online Model Distillation for Efficient Video Inference. In ICCV.

[33]

Feiyang Pan, Shuokai Li, Xiang Ao, Pingzhong Tang, and Qing He. 2019b. Warm Up Cold-Start Advertisements: Improving CTR Predictions via Learning to Learn ID Embeddings. In SIGIR.

[34]

Yiteng Pan, Fazhi He, and Haiping Yu. 2019a. A novel enhanced collaborative autoencoder with knowledge distillation for top-N recommender systems. Neurocomputing, Vol. 332 (2019), 137--148.

Digital Library

[35]

Zhimao Peng, Zechao Li, Junge Zhang, Yan Li, Guo-Jun Qi, and Jinhui Tang. 2019. Few-Shot Image Recognition With Knowledge Transfer. In ICCV.

[36]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In UAI.

Digital Library

[37]

Shaoyun Shi, Min Zhang, Xinxing Yu, Yongfeng Zhang, Bin Hao, Yiqun Liu, and Shaoping Ma. 2019. Adaptive Feature Sampling for Recommendation with Missing Content Feature Values. In CIKM.

[38]

Konstantin Shmelkov, Cordelia Schmid, and Karteek Alahari. 2017. Incremental Learning of Object Detectors Without Catastrophic Forgetting. In ICCV.

[39]

Changfeng Sun, Han Liu, Meng Liu, Zhaochun Ren, Tian Gan, and Liqiang Nie. 2020. LARA: Attribute-to-feature Adversarial Learning for New-item Recommendation. In WSDM.

Digital Library

[40]

Jiaxi Tang and Ke Wang. 2018. Ranking Distillation: Learning Compact Ranking Models With High Performance for Recommender System. In KDD.

Digital Library

[41]

John C. Tang, Gina Venolia, and Kori M. Inkpen. 2016. Meerkat and Periscope: I Stream, You Stream, Apps Stream for Live Streams. In CHI.

[42]

Gregor Urban, Krzysztof J. Geras, Samira Ebrahimi Kahou, Ozlem Aslan, Shengjie Wang, Abdelrahman Mohamed, Matthai Philipose, Matt Richardson, and Rich Caruana. 2017. Do Deep Convolutional Nets Really Need to be Deep and Convolutional?. In ICLR.

[43]

Aaron van den Oord, Sander Dieleman, and Benjamin Schrauwen. 2013. Deep content-based music recommendation. In NeurIPS.

[44]

Maksims Volkovs, Guang Wei Yu, and Tomi Poutanen. 2017. DropoutNet: Addressing Cold Start in Recommender Systems. In NeurIPS.

Digital Library

[45]

Haoyu Wang, Defu Lian, and Yong Ge. 2019b. Binarized collaborative filtering with distilling graph convolutional networks. arXiv preprint arXiv:1906.01829 (2019).

[46]

Shuai Wang, Kun Zhang, Le Wu, Haiping Ma, Richang Hong, and Meng Wang. 2021. Privileged Graph Distillation for Cold Start Recommendation. In SIGIR.

[47]

Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019a. Neural Graph Collaborative Filtering. In SIGIR.

[48]

Xinghua Wang, Zhaohui Peng, Senzhang Wang, Philip S Yu, Wenjing Fu, Xiaokang Xu, and Xiaoguang Hong. 2020. CDLFM: cross-domain recommendation for cold-start users via latent feature mapping. Knowledge and Information Systems, Vol. 62 (2020).

Digital Library

[49]

Xiaojie Wang, Rui Zhang, Yu Sun, and Jianzhong Qi. 2018. Kdgan: Knowledge distillation with generative adversarial networks. NeurIPS (2018).

[50]

Yinwei Wei, Xiang Wang, Qi Li, Liqiang Nie, Yan Li, Xuanping Li, and Tat-Seng Chua. 2021. Contrastive Learning for Cold-Start Recommendation. In MM.

[51]

Chen Xu, Quan Li, Junfeng Ge, Jinyang Gao, Xiaoyong Yang, Changhua Pei, Fei Sun, Jian Wu, Hanxiao Sun, and Wenwu Ou. 2020. Privileged Features Distillation at Taobao Recommendations. In KDD.

[52]

Xiaoxiao Xu, Chen Yang, Qian Yu, Zhiwei Fang, Jiaxing Wang, Chaosheng Fan, Yang He, Changping Peng, Zhangang Lin, and Jingping Shao. 2022. Alleviating Cold-Start Problem in CTR Prediction with A Variational Embedding Learning Framework. In WWW.

[53]

Peiyan Zhang, Jiayan Guo, Chaozhuo Li, Yueqi Xie, Jae Boum Kim, Yan Zhang, Xing Xie, Haohan Wang, and Sunghun Kim. 2023. Efficiently Leveraging Multi-level User Intent for Session-based Recommendation via Atten-Mixer Network. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining.

Digital Library

[54]

Yiding Zhang, Chaozhuo Li, Xing Xie, Xiao Wang, Chuan Shi, Yuming Liu, Hao Sun, Liangjie Zhang, Weiwei Deng, and Qi Zhang. 2022. Geometric Disentangled Collaborative Filtering. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval.

Digital Library

[55]

Yuan Zhang, Xiaoran Xu, Hanning Zhou, and Yan Zhang. 2020. Distilling structured knowledge into embeddings for explainable and accurate recommendation. In Proceedings of the 13th international conference on web search and data mining.

Digital Library

[56]

Jun Zhao, Zhou Zhou, Ziyu Guan, Wei Zhao, Wei Ning, Guang Qiu, and Xiaofei He. 2019. IntentGC: A Scalable Graph Convolution Framework Fusing Heterogeneous Information for Recommendation. In KDD.

[57]

Chunting Zhou, Jiatao Gu, and Graham Neubig. 2020. Understanding Knowledge Distillation in Non-autoregressive Machine Translation. In ICLR.

[58]

Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep Interest Evolution Network for Click-Through Rate Prediction. In AAAI.

[59]

Guorui Zhou, Xiaoqiang Zhu, Chengru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep Interest Network for Click-Through Rate Prediction. In KDD.

[60]

Huachi Zhou, Jiaqi Fan, Xiao Huang, Ka Ho Li, Zhenyu Tang, and Dahai Yu. 2022. Multi-Interest Refinement by Collaborative Attributes Modeling for Click-Through Rate Prediction. In CIKM.

[61]

Huachi Zhou, Qiaoyu Tan, Xiao Huang, Kaixiong Zhou, and Xiaoling Wang. 2021. Temporal augmented graph neural networks for session-based recommendations. In SIGIR.

[62]

Yongchun Zhu, Ruobing Xie, Fuzhen Zhuang, Kaikai Ge, Ying Sun, Xu Zhang, Leyu Lin, and Juan Cao. 2021. Learning to Warm Up Cold Item Embeddings for Cold-Start Recommendation with Meta Scaling and Shifting Networks. In SIGIR.

[63]

Ziwei Zhu, Shahin Sefati, Parsa Saadatpanah, and James Caverlee. 2020. Recommendation for New Users and New Items via Randomized Training and Mixture-of-Experts Transformation. In SIGIR.

Cited By

Xu XDong HXiang HHu XLi XXia XZhang XQi LDou W(2025)C2lRec: Causal Contrastive Learning for User Cold-start Recommendation with Social VariableACM Transactions on Information Systems10.1145/3711858Online publication date: 9-Jan-2025
https://dl.acm.org/doi/10.1145/3711858
Pan JCaverlee JZhu ZNejdl WAuer SKarras OCha MMoens MNajork M(2025)Combating Heterogeneous Model Biases in Recommendations via BoostingProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining10.1145/3701551.3703505(222-231)Online publication date: 10-Mar-2025
https://dl.acm.org/doi/10.1145/3701551.3703505
Bai YXian H(2025)Automatic Classification and Recommendation for English Teaching Materials Based on Natural Language ProcessingJournal of Circuits, Systems and Computers10.1142/S021812662550149XOnline publication date: 24-Feb-2025
https://doi.org/10.1142/S021812662550149X
Show More Cited By

Index Terms

Aligning Distillation For Cold-start Item Recommendation
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
      1. Social recommendation
2. Information systems
  1. World Wide Web
    1. Web searching and information discovery
      1. Social recommendation

Recommendations

Contrastive Collaborative Filtering for Cold-Start Item Recommendation
WWW '23: Proceedings of the ACM Web Conference 2023

The cold-start problem is a long-standing challenge in recommender systems. As a promising solution, content-based generative models usually project a cold-start item’s content onto a warm-start item embedding to capture collaborative signals from item ...
Generative Adversarial Framework for Cold-Start Item Recommendation
SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

The cold-start problem has been a long-standing issue in recommendation. Embedding-based recommendation models provide recommendations by learning embeddings for each user and item from historical interactions. Therefore, such embedding-based models ...
Contrastive Learning for Cold-Start Recommendation
MM '21: Proceedings of the 29th ACM International Conference on Multimedia

Recommending purely cold-start items is a long-standing and fundamental challenge in the recommender systems. Without any historical interaction on cold-start items, the collaborative filtering (CF) scheme fails to leverage collaborative signals to infer ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2023

3567 pages

ISBN:9781450394086

DOI:10.1145/3539618

General Chairs:
Hsin-Hsi Chen
National Taiwan University
,
Wei-Jou (Edward) Duh
National Taiwan University
,
Hen-Hsen Huang
Academia Sinica
,
Program Chairs:
Makoto P. Kato
Spotify
,
Josiane Mothe
Universite de Toulouse
,
Barbara Poblete
University of Chile and Amazon Visiting Academic

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China

Conference

SIGIR '23

Sponsor:

SIGIR

SIGIR '23: The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 23 - 27, 2023

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

66
Total Citations
View Citations
1,122
Total Downloads

Downloads (Last 12 months)705
Downloads (Last 6 weeks)67

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xu XDong HXiang HHu XLi XXia XZhang XQi LDou W(2025)C2lRec: Causal Contrastive Learning for User Cold-start Recommendation with Social VariableACM Transactions on Information Systems10.1145/3711858Online publication date: 9-Jan-2025
https://dl.acm.org/doi/10.1145/3711858
Pan JCaverlee JZhu ZNejdl WAuer SKarras OCha MMoens MNajork M(2025)Combating Heterogeneous Model Biases in Recommendations via BoostingProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining10.1145/3701551.3703505(222-231)Online publication date: 10-Mar-2025
https://dl.acm.org/doi/10.1145/3701551.3703505
Bai YXian H(2025)Automatic Classification and Recommendation for English Teaching Materials Based on Natural Language ProcessingJournal of Circuits, Systems and Computers10.1142/S021812662550149XOnline publication date: 24-Feb-2025
https://doi.org/10.1142/S021812662550149X
Yue HXia JZhang JPark JXie X(2025)Event-based adaptive fixed-time optimal control for saturated fault-tolerant nonlinear multiagent systems via reinforcement learning algorithmNeural Networks10.1016/j.neunet.2024.106952183(106952)Online publication date: Mar-2025
https://doi.org/10.1016/j.neunet.2024.106952
Wan SYang SFu Z(2025)Focus on user micro multi-behavioral statesInformation Processing and Management: an International Journal10.1016/j.ipm.2024.10396762:2Online publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1016/j.ipm.2024.103967
Wang YGuo CMa YFeng Q(2025)Recommendation feedback-based dynamic adaptive training for efficient social item recommendationExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.125605262:COnline publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1016/j.eswa.2024.125605
Zhang VWang GArshad NNguyen QXia MInnab N(2025)Advancing consumer insights: efficient sentiment analysis-based recommendation system using multi-criteria decision making with a game theoretic approachAnnals of Operations Research10.1007/s10479-024-06403-9Online publication date: 3-Jan-2025
https://doi.org/10.1007/s10479-024-06403-9
Zhao YHua X(2024)Application of collaborative filtering algorithm based on time decay function in music teaching recommendation modelPeerJ Computer Science10.7717/peerj-cs.253310(e2533)Online publication date: 26-Nov-2024
https://doi.org/10.7717/peerj-cs.2533
Massenon RGambo IOgundokun ROgundepo ESrivastava SAgarwal SPak W(2024)Mobile app review analysis for crowdsourcing of software requirements: a mapping study of automated and semi-automated toolsPeerJ Computer Science10.7717/peerj-cs.240110(e2401)Online publication date: 5-Nov-2024
https://doi.org/10.7717/peerj-cs.2401
Rehman AAhmed AAlahmadi TMirdad AAl Ghofaily BSaleem K(2024)A framework for generating recommendations based on trust in an informal e-learning environmentPeerJ Computer Science10.7717/peerj-cs.238610(e2386)Online publication date: 31-Oct-2024
https://doi.org/10.7717/peerj-cs.2386
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten