Ming Li
ABSTRACT
In several recommendation scenarios, including next basket recommendation, the importance of repetition and exploration has been discovered and studied. Sequential recommenders (SR) aim to infer a user's preferences and suggest the next item for them to interact with based on their historical interaction sequences. There has not been a systematic analysis of sequential recommenders from the perspective of repetition and exploration. As a result, it is unclear how these models, that are typically optimized for accuracy, perform in terms of repetition and exploration, as well as the potential drawbacks of deploying them in real applications.
In this paper, we examine whether repetition and exploration are important dimensions in the sequential recommendation scenario. We consider this generalizability question both from a user-centered and an item-centered perspective. Towards the latter, we define item repeat exposure and item explore exposure and examine the recommendation performance of sequential recommendation models in terms of both accuracy and exposure from the perspective of repetition and exploration. We find that (i) there is an imbalance in accuracy and difficulty w.r.t. repetition and exploration in SR scenarios, (ii) using the conventional average overall accuracy with a significance test does not fully represent a model's recommendation accuracy, and (iii) accuracy-oriented sequential recommendation models may suffer from less/zero item explore exposure issue, where items are mostly (or even only) recommended to their repeat users and fail to reach their potential new users.
To analyze our findings, we remove repeat samples from the dataset, that often act as easy shortcuts, and focus on a pure exploration SR scenario. We find that (i) removing the repetition shortcut increases the recommendation novelty and helps users who prefer to consume novel items next, (ii) neural-based models fail to learn the basic characteristics of this pure exploration scenario and suffer from an inherent repetitive bias issue, (iii) using shared item embeddings in the prediction layer may skew recommendations to repeat items, and (iv) removing all repeat items to post-processing recommendation results leads to a substantial improvement on top of several SR methods.
- Ashton Anderson, Ravi Kumar, Andrew Tomkins, and Sergei Vassilvitskii. 2014. The Dynamics of Repeat Consumption. In Proceedings of the 23rd International Conference on World Wide Web. 419--430.Google Scholar
Digital Library
- Mozhdeh Ariannezhad, Sami Jullien, Ming Li, Min Fang, Sebastian Schelter, and Maarten de Rijke. 2022. ReCANet: A Repeat Consumption-Aware Neural Network for Next Basket Recommendation in Grocery Shopping. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1240--1250.Google ScholarDigital Library
- Austin R Benson, Ravi Kumar, and Andrew Tomkins. 2016. Modeling User Consumption Sequences. In Proceedings of the 25th International Conference on World Wide Web. 519--529.Google ScholarDigital Library
- Jesús Bobadilla, Raúl Lara-Cabrera, Ángel González-Prieto, and Fernando Ortega. 2020. Deepfair: Deep Learning for Improving Fairness in Recommender Systems. arXiv preprint arXiv:2006.05255 (2020).Google Scholar
- Jun Chen, Chaokun Wang, Jianmin Wang, and S Yu Philip. 2016. Recommendation for Repeat Consumption from User Implicit Feedback. IEEE Transactions on Knowledge and Data Engineering, Vol. 28, 11 (2016), 3083--3097.Google ScholarDigital Library
- Wanyu Chen, Pengjie Ren, Fei Cai, Fei Sun, and Maarten de Rijke. 2020b. Improving End-to-end Sequential Recommendations with Intent-aware Diversification. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 175--184.Google ScholarDigital Library
- Zhilong Chen, Hancheng Cao, Huangdong Wang, Fengli Xu, Vassilis Kostakos, and Yong Li. 2020a. Will You Come back/Check-in Again? Understanding Characteristics Leading to Urban Revisitation and Re-check-in. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 4. 1--27.Google ScholarDigital Library
- Michael D. Ekstrand, Robin Burke, and Fernando Diaz. 2019. Fairness and Discrimination in Retrieval and Recommendation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1403--1404.Google Scholar
- Hui Fang, Danning Zhang, Yiheng Shu, and Guibing Guo. 2020. Deep Learning for Sequential Recommendation: Algorithms, Influential Factors, and Evaluations. ACM Transactions on Information Systems, Vol. 39, 1 (2020), Article 10.Google ScholarDigital Library
- Yingqiang Ge, Shuchang Liu, Ruoyuan Gao, Yikun Xian, Yunqi Li, Xiangyu Zhao, Changhua Pei, Fei Sun, Junfeng Ge, Wenwu Ou, and Yongfeng Zhang. 2021. Towards Long-Term Fairness in Recommendation. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 445--453.Google ScholarDigital Library
- Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. 2020. Shortcut Learning in Deep Neural Networks. Nature Machine Intelligence, Vol. 2, 11 (2020), 665--673.Google ScholarCross Ref
- Balázs Hidasi and Alexandros Karatzoglou. 2018. Recurrent Neural Networks with Top-k Gains for Session-based Recommendations. In Proceedings of the 27th ACM International Conference on Information & Knowledge Management. 843--852.Google ScholarDigital Library
- Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015. Session-based Recommendations with Recurrent Neural Networks. arXiv preprint arXiv:1511.06939 (2015).Google Scholar
- Yupeng Hou, Binbin Hu, Zhiqiang Zhang, and Wayne Xin Zhao. 2022. Core: Simple and Effective Session-based Recommendation within Consistent Representation Space. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval. 1796--1801.Google ScholarDigital Library
- Dietmar Jannach and Malte Ludewig. 2017. When Recurrent Neural Networks Meet the Neighborhood for Session-based Recommendation. In Proceedings of the Eleventh ACM Conference on Recommender Systems. 306--310.Google ScholarDigital Library
- Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive Sequential Recommendation. In 2018 IEEE International Conference on Data Mining. 197--206.Google Scholar
- Ori Katz, Oren Barkan, Noam Koenigstein, and Nir Zabari. 2022. Learning to Ride a Buy-Cycle: A Hyper-Convolutional Model for Next Basket Repurchase Recommendation. In Proceedings of the 16th ACM Conference on Recommender Systems. 316--326.Google ScholarDigital Library
- Duc-Trong Le, Hady W Lauw, and Yuan Fang. 2019. Correlation-sensitive Next-basket Recommendation. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2808--2814.Google ScholarCross Ref
- Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural Attentive Session-based Recommendation. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 1419--1428.Google ScholarDigital Library
- Ming Li, Sami Jullien, Mozhdeh Ariannezhad, and Maarten de Rijke. 2023. A Next Basket Recommendation Reality Check. ACM Transactions on Information Systems, Vol. 41, 4 (2023), Article 116.Google ScholarDigital Library
- Qiao Liu, Yifu Zeng, Refuoe Mokhosi, and Haibin Zhang. 2018. STAMP: Short-term Attention/Memory Priority Model for Session-based Recommendation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1831--1839.Google ScholarDigital Library
- Malte Ludewig and Dietmar Jannach. 2018. Evaluation of Session-based Recommendation Algorithms. User Modeling and User-Adapted Interaction, Vol. 28 (2018), 331--390.Google ScholarDigital Library
- Marco Morik, Ashudeep Singh, Jessica Hong, and Thorsten Joachims. 2020. Controlling Fairness and Bias in Dynamic Learning-to-Rank. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 429--438.Google ScholarDigital Library
- Mohammadmehdi Naghiaei, Hossein A. Rahmani, and Yashar Deldjoo. 2022. CPFair: Personalized Consumer and Producer Fairness Re-Ranking for Recommender Systems. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 770--779.Google ScholarDigital Library
- Aleksandr Petrov and Craig Macdonald. 2022. A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation. In Proceedings of the 16th ACM Conference on Recommender Systems. 436--447.Google ScholarDigital Library
- Mingda Qian, Xiaoyan Gu, Lingyang Chu, Feifei Dai, Haihui Fan, and Bo Li. 2022. Flexible Order Aware Sequential Recommendation. In Proceedings of the 2022 International Conference on Multimedia Retrieval. 109--117.Google ScholarDigital Library
- Massimo Quadrana, Paolo Cremonesi, and Dietmar Jannach. 2018. Sequence-Aware Recommender Systems. Comput. Surveys, Vol. 51, 4 (2018), 1--36.Google ScholarDigital Library
- Pengjie Ren, Zhumin Chen, Jing Li, Zhaochun Ren, Jun Ma, and Maarten de Rijke. 2019. RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-based Recommendation. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 4806--4813.Google ScholarDigital Library
- Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2008. The Graph Neural Network Model. IEEE Transactions on Neural Networks, Vol. 20, 1 (2008), 61--80.Google ScholarDigital Library
- Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1441--1450.Google ScholarDigital Library
- Jiaxi Tang and Ke Wang. 2018. Personalized Top-n Sequential Recommendation via Convolutional Sequence Embedding. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. 565--573.Google ScholarDigital Library
- Wilson L Taylor. 1953. "Cloze Procedure": A New Tool for Measuring Readability. Journalism Quarterly, Vol. 30, 4 (1953), 415--433.Google ScholarCross Ref
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. arXiv preprint arXiv:1706.03762 (2017).Google Scholar
- Shoujin Wang, Liang Hu, Yan Wang, Longbing Cao, Quan Z Sheng, and Mehmet Orgun. 2019a. Sequential Recommender Systems: Challenges, Progress and Prospects. arXiv preprint arXiv:2001.04830 (2019).Google Scholar
- Shoujin Wang, Liang Hu, Yan Wang, Quan Z Sheng, Mehmet Orgun, and Longbing Cao. 2019b. Modeling Multi-purpose Sessions for Next-item Recommendations via Mixture-channel Purpose Routing Networks. In International Joint Conference on Artificial Intelligence. 3771--3777.Google ScholarCross Ref
- Haolun Wu, Bhaskar Mitra, Chen Ma, Fernando Diaz, and Xue Liu. 2022. Joint Multisided Exposure Fairness for Recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 703--714.Google ScholarDigital Library
- Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019. Session-based Recommendation with Graph Neural Networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 346--353.Google ScholarDigital Library
- Yao Wu, Jian Cao, Guandong Xu, and Yudong Tan. 2021. TFROM: A Two-Sided Fairness-Aware Recommendation Model for Both Customers and Providers. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1013--1022.Google ScholarDigital Library
- Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Jiandong Zhang, Bolin Ding, and Bin Cui. 2022. Contrastive Learning for Sequential Recommendation. In 2022 IEEE 38th International Conference on Data Engineering. 1259--1273.Google Scholar
- Qing Yin, Hui Fang, Zhu Sun, and Yew-Soon Ong. 2022. Understanding Diversity in Session-Based Recommendation. arXiv preprint arXiv:2208.13453 (2022).Google Scholar
- Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Jundong Li, and Zi Huang. 2022. Self-Supervised Learning for Recommender Systems: A Survey. arXiv preprint arXiv:2203.15876 (2022).Google Scholar
- Meike Zehlike and Carlos Castillo. 2020. Reducing Disparate Exposure in Ranking: A Learning To Rank Approach. In Proceedings of The Web Conference 2020. 2849--2855.Google ScholarDigital Library
- Mi Zhang and Neil Hurley. 2008. Avoiding Monotony: Improving the Diversity of Recommendation Lists. In Proceedings of the 2008 ACM conference on Recommender systems. 123--130.Google ScholarDigital Library
- Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep Learning Based Recommender System: A Survey and New Perspectives. Comput. Surveys, Vol. 52, 1 (2019), 1--38.Google Scholar
- Wayne Xin Zhao, Junhua Chen, Pengfei Wang, Qi Gu, and Ji-Rong Wen. 2020. Revisiting Alternative Experimental Settings for Evaluating Top-n Item Recommendation Algorithms. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2329--2332.Google ScholarDigital Library
Index Terms
- Repetition and Exploration in Sequential Recommendation
Recommendations
User Popularity Preference Aware Sequential Recommendation
Computational Science – ICCS 2023AbstractIn recommender systems, users’ preferences for item popularity are diverse and dynamic, which reveals the different items that users prefer. Therefore, identifying user popularity preferences are significant for personalized recommendations. ...
Sequential Recommendation with Dual Side Neighbor-based Collaborative Relation Modeling
WSDM '20: Proceedings of the 13th International Conference on Web Search and Data MiningSequential recommendation task aims to predict user preference over items in the future given user historical behaviors. The order of user behaviors implies that there are resourceful sequential patterns embedded in the behavior history which reveal the ...
Improving Sequential Recommendation with Attribute-Augmented Graph Neural Networks
Advances in Knowledge Discovery and Data MiningAbstractMany practical recommender systems provide item recommendation for different users only via mining user-item interactions but totally ignoring the rich attribute information of items that users interact with. In this paper, we propose an attribute-...
Comments