research-article

A Generic Learning Framework for Sequential Recommendation with Distribution Shifts

Authors:

Xiang WangAuthors Info & Claims

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 331 - 340

https://doi.org/10.1145/3539618.3591624

Published: 18 July 2023 Publication History

Abstract

Leading sequential recommendation (SeqRec) models adopt empirical risk minimization (ERM) as the learning framework, which inherently assumes that the training data (historical interaction sequences) and the testing data (future interactions) are drawn from the same distribution. However, such i.i.d. assumption hardly holds in practice, due to the online serving and dynamic nature of recommender system.For example, with the streaming of new data, the item popularity distribution would change, and the user preference would evolve after consuming some items. Such distribution shifts could undermine the ERM framework, hurting the model's generalization ability for future online serving.

In this work, we aim to develop a generic learning framework to enhance the generalization of recommenders in the dynamic environment. Specifically, on top of ERM, we devise a Distributionally Robust Optimization mechanism for SeqRec (DROS). At its core is our carefully-designed distribution adaption paradigm, which considers the dynamics of data distribution and explores possible distribution shifts between training and testing. Through this way, we can endow the backbone recommenders with better generalization ability.It is worth mentioning that DROS is an effective model-agnostic learning framework, which is applicable to general recommendation scenarios.Theoretical analyses show that DROS enables the backbone recommenders to achieve robust performance in future testing data.Empirical studies verify the effectiveness against dynamic distribution shifts of DROS. Codes are anonymously open-sourced at https://github.com/YangZhengyi98/DROS.

References

[1]

Aharon Ben-Tal, Dick den Hertog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Rennen. 2013. Robust Solutions of Optimization Problems Affected by Uncertain Probabilities. Manag. Sci. 59 (2013), 341--357.

Digital Library

[2]

Pedro Dalla Vecchia Chaves, Bruno L. Pereira, and Rodrygo L. T. Santos. 2022. Efficient Online Learning to Rank for Sequential Music Recommendation. In WWW. 2442--2450.

[3]

Chong Chen, Min Zhang, Yongfeng Zhang, Yiqun Liu, and Shaoping Ma. 2020. Efficient Neural Matrix Factorization without Sampling for Recommendation. TOIS 38 (2020), 14:1--14:28.

[4]

Yongjun Chen, Zhiwei Liu, Jia Li, Julian J. McAuley, and Caiming Xiong. 2022. Intent Contrastive Learning for Sequential Recommendation. In WWW. 2172--2182.

[5]

Chen Cheng, Haiqin Yang, Michael R Lyu, and Irwin King. 2013. Where you like to go next: Successive point-of-interest recommendation. In IJCAI. 2605--2611.

[6]

Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep Neural Networks for YouTube Recommendations. In RecSys. 191--198.

[7]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL. 4171--4186.

[8]

Sihao Ding, Fuli Feng, Xiangnan He, Jinqiu Jin, Wenjie Wang, Yong Liao, and Yongdong Zhang. 2022. Interpolative Distillation for Unifying Biased and Debiased Recommendation. In SIGIR. 40--49.

[9]

Xinyan Fan, Jianxun Lian, Wayne Xin Zhao, Zheng Liu, Chaozhuo Li, and Xing Xie. 2022. Ada-Ranker: A Data Distribution Adaptive Ranking Paradigm for Sequential Recommendation. In SIGIR. 1599--1610.

[10]

Xinyan Fan, Zheng Liu, Jianxun Lian, Wayne Xin Zhao, Xing Xie, and Ji-Rong Wen. 2021. Lighter and Better: Low-Rank Decomposed Self-Attention Networks for Next-Item Recommendation. In SIGIR. 1733--1737.

[11]

Chongming Gao, Shijun Li, Wenqiang Lei, Jiawei Chen, Biao Li, Peng Jiang, Xiangnan He, Jiaxin Mao, and Tat-Seng Chua. 2022. KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems. In CIKM.

[12]

Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick. 2022. Masked Autoencoders Are Scalable Vision Learners. In CVPR. 15979--15988.

[13]

Ruining He and Julian McAuley. 2016. Fusing similarity models with markov chains for sparse sequential recommendation. In ICDM. 191--200.

[14]

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based Recommendations with Recurrent Neural Networks. In ICLR.

[15]

Zhaolin Hu and L Jeff Hong. 2013. Kullback-Leibler divergence constrained distributionally robust optimization. Available at Optimization Online (2013), 1695--1724.

[16]

Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recom- mendation. In ICDM. 197--206.

[17]

David Krueger, Ethan Caballero, Jörn-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Rémi Le Priol, and Aaron C. Courville. 2021. Out-of- Distribution Generalization via Risk Extrapolation (REx). In ICML, Vol. 139. 5815--5826.

[18]

Henry Lam. 2016. Robust Sensitivity Analysis for Stochastic Systems. Math. Oper. Res. 41 (2016), 1248--1275.

Digital Library

[19]

Fengming Lin, Xiaolei Fang, and Zheming Gao. 2022. Distributionally Robust Optimization: A review on theory and applications. Numerical Algebra, Control & Optimization 12 (2022), 159.

[20]

Divyat Mahajan, Shruti Tople, and Amit Sharma. 2021. Domain generalization using causal matching. In ICML. 7313--7324.

[21]

Lech Maligranda. 1998. Why Hölder's inequality should be called Rogers' in- equality. Mathematical Inequalities & Applications 1 (1998), 69--83.

[22]

Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. 2012. Foundations of Machine Learning. MIT Press.

Digital Library

[23]

Yabo Ni, Dan Ou, Shichen Liu, Xiang Li, Wenwu Ou, Anxiang Zeng, and Luo Si. 2018. Perceive Your Users in Depth: Learning Universal User Representations from Multiple E-commerce Tasks. In SIGKDD. 596--605.

[24]

Yonatan Oren, Shiori Sagawa, Tatsunori B. Hashimoto, and Percy Liang. 2019. Distributionally Robust Language Modeling. In EMNLP. 4226--4236.

[25]

Aleksandr Petrov and Craig Macdonald. 2022. Effective and Efficient Training for Sequential Recommendation using Recency Sampling. In RecSys. ACM, 81--91.

[26]

Ruihong Qiu, Zi Huang, Hongzhi Yin, and Zijian Wang. 2022. Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation. In WSDM, K. Selcuk Candan, Huan Liu, Leman Akoglu, Xin Luna Dong, and Jiliang Tang (Eds.). 813--823.

[27]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In UAI. 452-- 461.

[28]

Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factor- izing personalized markov chains for next-basket recommendation. In WWW. 811--820.

[29]

Alexander Robey, George J Pappas, and Hamed Hassani. 2021. Model-based domain generalization. NeurIPS 34 (2021), 20210--20229.

[30]

Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. 2020. Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization. In ICLR.

[31]

Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as Treatments: Debiasing Learning and Evaluation. In ICML. JMLR.org, 1670--1679.

[32]

Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer. In CIKM. 1441--1450.

[33]

Qiaoyu Tan, Jianwei Zhang, Jiangchao Yao, Ninghao Liu, Jingren Zhou, Hongxia Yang, and Xia Hu. 2021. Sparse-Interest Network for Sequential Recommendation. In WSDM. 598--606.

[34]

Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In WSDM. 565--573.

[35]

Riccardo Volpi, Hongseok Namkoong, Ozan Sener, John C. Duchi, Vittorio Murino, and Silvio Savarese. 2018. Generalizing to Unseen Domains via Adversarial Data Augmentation. In NeurIPS. 5339--5349.

[36]

Qi Wan, Xiangnan He, Xiang Wang, Jiancan Wu, Wei Guo, and Ruiming Tang. 2022. Cross Pairwise Ranking for Unbiased Item Recommendation. In WWW. 2370--2378.

[37]

Xuejian Wang, Lantao Yu, Kan Ren, Guanyu Tao, Weinan Zhang, Yong Yu, and Jun Wang. 2017. Dynamic Attention Deep Model for Article Recommendation by Learning Human Editors' Demonstration. In SIGKDD. 2051--2059.

[38]

Zimu Wang, Yue He, Jiashuo Liu, Wenchao Zou, Philip S. Yu, and Peng Cui. 2022. Invariant Preference Learning for General Debiasing in Recommendation. In SIGKDD. 1969--1978.

[39]

Zijian Wang, Yadan Luo, Ruihong Qiu, Zi Huang, and Mahsa Baktashmotlagh. 2021. Learning to Diversify for Single Domain Generalization. In ICCV. 814--823.

[40]

Zhenlei Wang, Shiqi Shen, Zhipeng Wang, Bo Chen, Xu Chen, and Ji-Rong Wen. 2022. Unbiased Sequential Recommendation with Latent Confounders. In WWW. 2195--2204.

[41]

Hongyi Wen, Xinyang Yi, Tiansheng Yao, Jiaxi Tang, Lichan Hong, and Ed H. Chi. 2022. Distributionally-robust Recommendations for Improving Worst-case User Experience. In WWW. 3606--3610.

[42]

Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2021. Self-supervised Graph Learning for Recommendation. In SIGIR. ACM, 726--735.

[43]

Jiancan Wu, Xiang Wang, Xingyu Gao, Jiawei Chen, Hongcheng Fu, Tianyu Qiu, and Xiangnan He. 2022. On the Effectiveness of Sampled Softmax Loss for Item Recommendation. CoRR abs/2201.02327 (2022).

[44]

Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Jiandong Zhang, Bolin Ding, and Bin Cui. 2022. Contrastive Learning for Sequential Recommendation. In ICDE. 1259--1273.

[45]

Fajie Yuan, Xiangnan He, Alexandros Karatzoglou, and Liguang Zhang. 2020. Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation. In SIGIR. 1469--1478.

[46]

Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M. Jose, and Xiangnan He. 2019. A Simple Convolutional Generative Network for Next Item Recommendation. In WSDM. 582--590.

[47]

An Zhang, Wenchang Ma, Xiang Wang, and Tat seng Chua. 2022. Incorporating Bias-aware Margins into Contrastive Loss for Collaborative Filtering. In NeurIPS.

[48]

An Zhang, Jingnan Zheng, Xiang Wang, Yancheng Yuan, and Tat seng Chua. 2023. Invariant Collaborative Filtering to Popularity Distribution Shift. In WWW.

[49]

Yang Zhang, Fuli Feng, Xiangnan He, Tianxin Wei, Chonggang Song, Guohui Ling, and Yongdong Zhang. 2021. Causal Intervention for Leveraging Popularity Bias in Recommendation. In SIGIR. 11--20.

[50]

Long Zhao, Ting Liu, Xi Peng, and Dimitris N. Metaxas. 2020. Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness. In NeurIPS.

[51]

Shanshan Zhao, Mingming Gong, Tongliang Liu, Huan Fu, and Dacheng Tao. 2020. Domain Generalization via Entropy Regularization. In NeurIPS.

[52]

Yu Zheng, Chen Gao, Xiang Li, Xiangnan He, Yong Li, and Depeng Jin. 2021. Disentangling User Interest and Conformity for Recommendation with Causal Embedding. In WWW. 2980--2991.

[53]

Guorui Zhou, Xiaoqiang Zhu, Chengru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep Interest Network for Click-Through Rate Prediction. In SIGKDD. 1059--1068

Cited By

Tang MCui SJin ZLiang SLi CZou L(2025)Sequential recommendation by reprogramming pretrained transformerInformation Processing & Management10.1016/j.ipm.2024.10393862:1(103938)Online publication date: Jan-2025
https://doi.org/10.1016/j.ipm.2024.103938
Caroprese LPisani FVeloso BKonig MManco GHoos HGama J(2024)Modelling Concept Drift in Dynamic Data Streams for Recommender SystemsACM Transactions on Recommender Systems10.1145/37076933:2(1-28)Online publication date: 9-Dec-2024
https://dl.acm.org/doi/10.1145/3707693
Zhang ZWang XChen HLi HZhu W(2024)Disentangled Dynamic Graph Attention Network for Out-of-Distribution Sequential RecommendationACM Transactions on Information Systems10.1145/370198843:1(1-42)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3701988
Show More Cited By

Index Terms

A Generic Learning Framework for Sequential Recommendation with Distribution Shifts
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking
    2. Retrieval tasks and goals
      1. Recommender systems

Recommendations

User Popularity Preference Aware Sequential Recommendation
Computational Science – ICCS 2023
Abstract
In recommender systems, users’ preferences for item popularity are diverse and dynamic, which reveals the different items that users prefer. Therefore, identifying user popularity preferences are significant for personalized recommendations. ...
Multi-interest sequential recommendation with contrastive learning and temporal analysis
Abstract
Sequential recommendation systems aim to forecast the subsequent item of interest to users by analyzing their historical behaviors. While existing approaches, which employ attention mechanisms, have significantly advanced by capturing users’ ...
Highlights
- Proposing a multi-interest recommendation approach that incorporates user preference features and temporal characteristics at the user level.
- Introducing a time-period module to capture short-term changes in user interest.
- ...
APGL4SR: A Generic Framework with Adaptive and Personalized Global Collaborative Information in Sequential Recommendation
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

The sequential recommendation system has been widely studied for its promising effectiveness in capturing dynamic preferences buried in users' sequential behaviors. Despite the considerable achievements, existing methods usually focus on intra-sequence ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2023

3567 pages

ISBN:9781450394086

DOI:10.1145/3539618

General Chairs:
Hsin-Hsi Chen
National Taiwan University
,
Wei-Jou (Edward) Duh
National Taiwan University
,
Hen-Hsen Huang
Academia Sinica
,
Program Chairs:
Makoto P. Kato
Spotify
,
Josiane Mothe
Universite de Toulouse
,
Barbara Poblete
University of Chile and Amazon Visiting Academic

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

the National Natural Science Foundation of China
the University Synergy Innovation Program of Anhui Province
the National Key Research and Development Program of China

Conference

SIGIR '23

Sponsor:

SIGIR

SIGIR '23: The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 23 - 27, 2023

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
761
Total Downloads

Downloads (Last 12 months)391
Downloads (Last 6 weeks)37

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tang MCui SJin ZLiang SLi CZou L(2025)Sequential recommendation by reprogramming pretrained transformerInformation Processing & Management10.1016/j.ipm.2024.10393862:1(103938)Online publication date: Jan-2025
https://doi.org/10.1016/j.ipm.2024.103938
Caroprese LPisani FVeloso BKonig MManco GHoos HGama J(2024)Modelling Concept Drift in Dynamic Data Streams for Recommender SystemsACM Transactions on Recommender Systems10.1145/37076933:2(1-28)Online publication date: 9-Dec-2024
https://dl.acm.org/doi/10.1145/3707693
Zhang ZWang XChen HLi HZhu W(2024)Disentangled Dynamic Graph Attention Network for Out-of-Distribution Sequential RecommendationACM Transactions on Information Systems10.1145/370198843:1(1-42)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3701988
Lin JPeng SZhang ZZhao P(2024)TLRec: A Transfer Learning Framework to Enhance Large Language Models for Sequential Recommendation TasksProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3691710(1119-1124)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3691710
Cui YLiu FWang PWang BTang HWan YWang JChen J(2024)Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language ModelsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688118(507-517)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688118
Lin XWang WLi YFeng FNg SChua TBaeza-Yates RBonchi F(2024)Bridging Items and Language: A Transition Paradigm for Large Language Model-Based RecommendationProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671884(1816-1826)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671884
Wu JWang XGao XChen JFu HQiu T(2024)On the Effectiveness of Sampled Softmax Loss for Item RecommendationACM Transactions on Information Systems10.1145/363706142:4(1-26)Online publication date: 22-Mar-2024
https://dl.acm.org/doi/10.1145/3637061
Liao JLi SYang ZWu JYuan YWang XHe XHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)LLaRA: Large Language-Recommendation AssistantProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657690(1785-1795)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657690
Yang JDing YWang YRen PChen ZCai FMa JZhang RRen ZXin XAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)Debiasing Sequential Recommenders through Distributionally Robust Optimization over System ExposureProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635848(882-890)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635848
Jiang MBao KZhang JWang WYang ZFeng FHe XChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Item-side Fairness of Large Language Model-based Recommendation SystemProceedings of the ACM Web Conference 202410.1145/3589334.3648158(4717-4726)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3648158
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten