tutorial

Large Language Models for Recommendation: Progresses and Future Directions

Authors:

Xiangnan HeAuthors Info & Claims

SIGIR-AP '23: Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region

Pages 306 - 309

https://doi.org/10.1145/3624918.3629550

Published: 26 November 2023 Publication History

Abstract

The powerful large language models (LLMs) have played a pivotal role in advancing recommender systems. Recently, in both academia and industry, there has been a surge of interest in developing LLMs for recommendation, referred to as LLM4Rec. This includes endeavors like leveraging LLMs for generative item retrieval and ranking, as well as the exciting possibility of building universal LLMs for diverse open-ended recommendation tasks. These developments hold the potential to reshape the traditional recommender paradigm, paving the way for the next-generation recommender systems.

In this tutorial, we aim to retrospect the evolution of LLM4Rec and conduct a comprehensive review of existing research. In particular, we will clarify how recommender systems benefit from LLMs through a variety of perspectives, including the model architecture, learning paradigm, and the strong abilities of LLMs such as chatting, generalization, planning, and generation. Furthermore, we will discuss the critical challenges and open problems in this emerging field, for instance, the trustworthiness, efficiency, and model retraining issues. Lastly, we will summarize the implications of previous work and outline future research directions. We believe that this tutorial will assist the audience in better understanding the progress and prospects of LLM4Rec, inspiring them for future exploration. This, in turn, will drive the prosperity of LLM4Rec, possibly fostering a paradigm shift in recommendation systems.

References

[1]

Qingyao Ai 2023. Information Retrieval Meets Large Language Models: A Strategic Report from Chinese IR Community. AI Open 4 (2023), 80–90.

[2]

Rohan Anil 2023. Palm 2 technical report. arXiv preprint arXiv:2305.10403 (2023).

[3]

Yuntao Bai 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073 (2022).

[4]

Yuntao Bai 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 (2022).

[5]

Keqin Bao, Jizhi Zhang, Wenjie Wang, Yang Zhang, Zhengyi Yang, Yancheng Luo, Fuli Feng, Xiangnan He, and Qi Tian. 2023. A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systems. CoRR abs/2308.08434 (2023). https://doi.org/10.48550/arXiv.2308.08434 arXiv:2308.08434

[6]

Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. 2023. TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation. In RecSys. ACM, 1007–1014. https://doi.org/10.1145/3604915.3608857

Digital Library

[7]

Lukas Berglund 2023. The Reversal Curse: LLMs trained on “A is B” fail to learn “B is A”. arXiv preprint arXiv:2309.12288 (2023).

[8]

Steven Bills 2023. Language models can explain neurons in language models. URL https://openaipublic. blob. core. windows. net/neuron-explainer/paper/index. html.(Date accessed: 14.05. 2023) (2023).

[9]

Boxi Cao 2023. The Life Cycle of Knowledge in Big Language Models: A Survey. arXiv preprint arXiv:2303.07616 (2023).

[10]

Aldo Gael Carranza 2023. Privacy-Preserving Recommender Systems with Synthetic Query Generation using Differentially Private Large Language Models. arXiv preprint arXiv:2305.05973 (2023).

[11]

Jiawei Chen, Xiang Wang, Fuli Feng, and Xiangnan He. 2021. Bias Issues and Solutions in Recommender System. In RecSys. 825–827.

[12]

Wuyang Chen 2023. Lifelong language pretraining with distribution-specialized experts. In ICML. PMLR, 5383–5395.

[13]

Xu Chen 2023. REASONER: An Explainable Recommendation Dataset with Multi-aspect Real User Labeled Ground Truths Towards more Measurable Explainable Recommendation. arXiv preprint arXiv:2303.00168 (2023).

[14]

Zeyu Cui 2022. M6-rec: Generative pretrained language models are open-ended recommender systems. arXiv preprint arXiv:2205.08084 (2022).

[15]

Sunhao Dai 2023. Uncovering ChatGPT’s Capabilities in Recommender Systems. In RecSys, Jie Zhang, Li Chen, Shlomo Berkovsky, Min Zhang, Tommaso Di Noia, Justin Basilico, Luiz Pizzato, and Yang Song (Eds.). ACM, 1126–1132.

[16]

Tri Dao 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems 35 (2022), 16344–16359.

[17]

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314 (2023).

[18]

Nan Du 2022. Glam: Efficient scaling of language models with mixture-of-experts. In ICML. PMLR, 5547–5569.

[19]

Wenqi Fan, Zihuai Zhao, Jiatong Li, Yunqing Liu, Xiaowei Mei, Yiqi Wang, Jiliang Tang, and Qing Li. 2023. Recommender systems in the era of large language models (llms). arXiv preprint arXiv:2307.02046 (2023).

[20]

Yue Feng 2023. A Large Language Model Enhanced Conversational Recommender System. arXiv preprint arXiv:2308.06212 (2023).

[21]

Luke Friedman 2023. Leveraging Large Language Models in Conversational Recommender Systems. arXiv preprint arXiv:2305.07961 (2023).

[22]

Yunfan Gao 2023. Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System. CoRR abs/2303.14524 (2023).

[23]

Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022. Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5). In RecSys. 299–315.

[24]

Xiangnan He 2017. Neural Collaborative Filtering. In WWW, Rick Barrett, Rick Cummings, Eugene Agichtein, and Evgeniy Gabrilovich (Eds.). ACM, 173–182.

[25]

Xiangnan He 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In SIGIR. 639–648.

[26]

Xiangnan He and Tat-Seng Chua. 2017. Neural Factorization Machines for Sparse Predictive Analytics. In SIGIR. ACM, 355–364.

[27]

Zhankui He 2023. Large language models as zero-shot conversational recommenders. arXiv preprint arXiv:2308.10053 (2023).

[28]

Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. 2022. Towards universal sequence representation learning for recommender systems. In SIGKDD. 585–593.

[29]

Wenyue Hua, Lei Li, Shuyuan Xu, Li Chen, and Yongfeng Zhang. 2023. Tutorial on Large Language Models for Recommendation. In RecSys. 1281–1283.

[30]

Wang-Cheng Kang 2023. Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction. CoRR abs/2305.06474 (2023).

[31]

Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In ICDM. IEEE, 197–206.

[32]

Woosuk Kwon 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. arXiv preprint arXiv:2309.06180 (2023).

[33]

Wenqiang Lei, Xiangnan He, Maarten de Rijke, and Tat-Seng Chua. 2020. Conversational recommendation: Formulation, methods, and evaluation. In SIGIR. 2425–2428.

[34]

Jinming Li 2023. GPT4Rec: A generative framework for personalized recommendation and user interests interpretation. arXiv preprint arXiv:2304.03879 (2023).

[35]

Jiacheng Li 2023. Text Is All You Need: Learning Language Representations for Sequential Recommendation. In SIGKDD. ACM, 1258–1267.

[36]

Jianghao Lin 2023. ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation. arXiv preprint arXiv:2308.11131 (2023).

[37]

Jianghao Lin, Xinyi Dai, Yunjia Xi, Weiwen Liu, Bo Chen, Xiangyang Li, Chenxu Zhu, Huifeng Guo, Yong Yu, Ruiming Tang, 2023. How Can Recommender Systems Benefit from Large Language Models: A Survey. arXiv preprint arXiv:2306.05817 (2023).

[38]

Junling Liu 2023. Is ChatGPT a Good Recommender? A Preliminary Study. CoRR abs/2304.10149 (2023).

[39]

OpenAI. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774 (2023).

[40]

Bo Peng 2023. RWKV: Reinventing RNNs for the Transformer Era. arXiv preprint arXiv:2305.13048 (2023).

[41]

Zhaopeng Qiu 2021. U-BERT: Pre-training User Representations for Improved Recommendation. In AAAI. AAAI Press, 4320–4327.

[42]

Zhaochun Ren, Xiangnan He, Dawei Yin, and Maarten de Rijke. 2018. Information Discovery in E-commerce: Half-day SIGIR 2018 Tutorial. In SIGIR. 1379–1382.

Digital Library

[43]

Zhiqiang Shen 2023. SlimPajama-DC: Understanding Data Combinations for LLM Training. arXiv preprint arXiv:2309.10818 (2023).

[44]

Giuseppe Spillo 2023. Towards Sustainability-aware Recommender Systems: Analyzing the Trade-off Between Algorithms Performance and Carbon Footprint. In RecSys. 856–862.

[45]

Fei Sun 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management. 1441–1450.

Digital Library

[46]

Jindong Wang 2023. On the robustness of chatgpt: An adversarial and out-of-distribution perspective. arXiv preprint arXiv:2302.12095 (2023).

[47]

Lei Wang 2023. RecAgent: A Novel Simulation Paradigm for Recommender Systems. arXiv preprint arXiv:2306.02552 (2023).

[48]

Lei Wang and Ee-Peng Lim. 2023. Zero-Shot Next-Item Recommendation using Large Pretrained Language Models. CoRR abs/2304.03153 (2023).

[49]

Wenjie Wang 2023. Generative recommendation: Towards next-generation recommender paradigm. arXiv preprint arXiv:2304.03516 (2023).

[50]

Xiang Wang 2019. Neural Graph Collaborative Filtering. In SIGIR, Benjamin Piwowarski, Max Chevalier, Éric Gaussier, Yoelle Maarek, Jian-Yun Nie, and Falk Scholer (Eds.). ACM, 165–174.

[51]

Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2020. Learning and reasoning on graph for recommendation. In WSDM. 890–893.

[52]

Xiaolei Wang, Xinyu Tang, Wayne Xin Zhao, Jingyuan Wang, and Ji-Rong Wen. 2023. Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models. arXiv preprint arXiv:2305.13112 (2023).

[53]

Yancheng Wang 2023. Recmind: Large language model powered agent for recommendation. arXiv preprint arXiv:2308.14296 (2023).

[54]

Yifan Wang 2023. A survey on the fairness of recommender systems. ACM Transactions on Information Systems 41, 3 (2023), 1–43.

Digital Library

[55]

Likang Wu 2023. Exploring Large Language Model for Graph Data Understanding in Online Job Recommendations. CoRR abs/2307.05722 (2023).

[56]

Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, 2023. A Survey on Large Language Models for Recommendation. arXiv preprint arXiv:2305.19860 (2023).

[57]

Yunjia Xi 2023. Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models. CoRR abs/2306.10933 (2023).

[58]

Zhiheng Xi 2023. The Rise and Potential of Large Language Model Based Agents: A Survey. arXiv preprint arXiv:2309.07864 (2023).

[59]

Jun Xu, Xiangnan He, and Hang Li. 2018. Deep learning for matching in search and recommendation. In SIGIR. 1365–1368.

[60]

Minrui Xu 2023. Unleashing the power of edge-cloud generative ai in mobile networks: A survey of aigc services. arXiv preprint arXiv:2303.16129 (2023).

[61]

Zheng Yuan 2023. Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisited. In SIGIR, Hsin-Hsi Chen, Wei-Jou (Edward) Duh, Hen-Hsen Huang, Makoto P. Kato, Josiane Mothe, and Barbara Poblete (Eds.). ACM, 2639–2649.

[62]

Junjie Zhang 2023. Recommendation as instruction following: A large language model empowered recommendation approach. arXiv preprint arXiv:2305.07001 (2023).

[63]

Jizhi Zhang, Keqin Bao, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. 2023. Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation. In RecSys. ACM, 993–999. https://doi.org/10.1145/3604915.3608860

Digital Library

[64]

Yuhui Zhang 2021. Language models as recommender systems: Evaluations and limitations. (2021).

[65]

Yongfeng Zhang, Xu Chen, 2020. Explainable recommendation: A survey and new perspectives. Foundations and Trends® in Information Retrieval 14, 1 (2020), 1–101.

Digital Library

[66]

Yang Zhang, Fuli Feng, Xiangnan He, Tianxin Wei, Chonggang Song, Guohui Ling, and Yongdong Zhang. 2021. Causal Intervention for Leveraging Popularity Bias in Recommendation. In SIGIR. 11–20.

[67]

Wayne Xin Zhao 2023. A survey of large language models. arXiv preprint arXiv:2303.18223 (2023).

Cited By

Seesukong SAngskun TKeandoungchun NThippongtorn AAngskun J(2024)CARIAInternational Journal of Information and Communication Technology Education10.4018/IJICTE.35649920:1(1-24)Online publication date: 14-Oct-2024
https://doi.org/10.4018/IJICTE.356499
Kolb T(2024)Enhancing Cross-Domain Recommender Systems with LLMs: Evaluating Bias and Beyond-Accuracy MeasuresProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688027(1388-1394)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688027
Pessach DPoblete B(2024)Gender Representation Across Online Retail ProductsProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658947(947-957)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658947
Show More Cited By

Index Terms

Large Language Models for Recommendation: Progresses and Future Directions
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Large Language Models for Recommendation: Progresses and Future Directions
WWW '24: Companion Proceedings of the ACM Web Conference 2024

Large language models (LLMs) have significantly influenced recommender systems. Both academia and industry have shown growing interest in developing LLMs for recommendation purposes, an approach commonly referred to as LLM4Rec. This involves efforts such ...
Large Language Models for Recommendation: Past, Present, and Future
SIGIR '24: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

Large language models (LLMs) have significantly influenced recommender systems, spurring interest across academia and industry in leveraging LLMs for recommendation tasks. This includes using LLMs for generative item retrieval and ranking, and developing ...
GenRec: Large Language Model for Generative Recommendation
Advances in Information Retrieval
Abstract
In recent years, Large Language Models (LLMs) have emerged as powerful tools for diverse natural language processing tasks. However, their potential for recommender systems under the generative recommendation paradigm remains relatively ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR-AP '23: Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region

November 2023

324 pages

ISBN:9798400704086

DOI:10.1145/3624918

Editors:
Qingyao Ai
Tsinghua University, China
,
Yiqin Liu
Tsinghua University, China
,
Alistair Moffat
The University of Melbourne, Australia
,
Xuanjing Huang
Fudan University, China
,
Tetsuya Sakai
Waseda University, Japan
,
Justin Zobel
The University of Melbourne, Australia

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 November 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Tutorial
Research
Refereed limited

Conference

SIGIR-AP '23

Sponsor:

SIGIR

SIGIR-AP '23: Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region

November 26 - 28, 2023

Beijing, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
1,189
Total Downloads

Downloads (Last 12 months)632
Downloads (Last 6 weeks)41

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Seesukong SAngskun TKeandoungchun NThippongtorn AAngskun J(2024)CARIAInternational Journal of Information and Communication Technology Education10.4018/IJICTE.35649920:1(1-24)Online publication date: 14-Oct-2024
https://doi.org/10.4018/IJICTE.356499
Kolb T(2024)Enhancing Cross-Domain Recommender Systems with LLMs: Evaluating Bias and Beyond-Accuracy MeasuresProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688027(1388-1394)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688027
Pessach DPoblete B(2024)Gender Representation Across Online Retail ProductsProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658947(947-957)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658947
Bao KZhang JLin XZhang YWang WFeng FHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Large Language Models for Recommendation: Past, Present, and FutureProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3661383(2993-2996)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3661383
Zhang ADeng YLin YChen XWen JChua THui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Large Language Model Powered Agents for Information RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3661375(2989-2992)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3661375
Li XLin LWang SQian CHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Self-Improving Teacher Cultivates Better Student: Distillation Calibration for Multimodal Large Language ModelsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657692(882-892)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657692
Jiang MBao KZhang JWang WYang ZFeng FHe XChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Item-side Fairness of Large Language Model-based Recommendation SystemProceedings of the ACM Web Conference 202410.1145/3589334.3648158(4717-4726)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3648158
Li YLiu KSatapathy RWang SCambria E(2024)Recent Developments in Recommender Systems: A Survey [Review Article]IEEE Computational Intelligence Magazine10.1109/MCI.2024.336398419:2(78-95)Online publication date: 8-Apr-2024
https://dl.acm.org/doi/10.1109/MCI.2024.3363984
Xiong ZLi HLiu ZChen ZZhou HRong WOuyang Y(2024)A Review of Data Mining in Personalized Education: Current Trends and Future ProspectsFrontiers of Digital Education10.1007/s44366-024-0019-61:1(26-50)Online publication date: 2-Jul-2024
https://doi.org/10.1007/s44366-024-0019-6
Zhang XZhao GRen YWang WCai WZhao YZhang XLiu J(2024)Data augmented large language models for medical record generationApplied Intelligence10.1007/s10489-024-05934-955:2Online publication date: 6-Dec-2024
https://doi.org/10.1007/s10489-024-05934-9
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten