Abstract
Neural network-based sequential recommendation models, such as SASRec and GRU4Rec, struggle with highly repetitive recommendations due to a problem known as the SoftMax bottleneck: it is hard to model a multi-modal probability distribution, which is common in repetitive recommendations, using a single-vector sequence representation. As a solution, a recently proposed state-of-the-art approach called CPR uses separate sub-models to compute scores for repeated and non-repeated interactions. However, CPR does not explicitly consider the counts of past repetitions, one of the most important signals for predicting future interactions, and counting repetitions in the models’ inputs is known to be a hard problem for neural networks. Building upon CPR ideas, we propose CountNet, which models repetition frequencies using Laplace and Dirichlet models; two simple yet efficient statistical approaches frequently used to describe probability distributions in highly-repetitive data. As a result, we improve the effectiveness of sequential recommendation models in repeated recommendation scenarios. For instance, on the Gowalla dataset, we improve Recall@10 of SASRec by 23% and by 5.96% when compared to SASRec enhanced by CPR. CountNet can be easily applied on top of many existing sequential recommendation models, and we show that these improvements generalize through extensive experiments with three different backbone models and three real-world sequential datasets with repeated interactions.
A. V. Petrov—Work done as an intern at Amazon.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Note that the data generation distribution is not directly observable, and the training data only has samples drawn from this distribution.
- 2.
In our experiments we avoid using recommendations datasets that have been criticised for not being sequential [11].
References
Abbattista, D., Anelli, V.W., Di Noia, T., Macdonald, C., Petrov, A.V.: Enhancing sequential music recommendation with personalized popularity awareness. In: Proceedings of the RecSys (2024)
Alba, J.W., Hutchinson, J.W.: Dimensions of consumer expertise. J. Consum. Res. 13(4), 411 (1987)
Anelli, V.W., et al.: Elliot: a comprehensive and rigorous framework for reproducible recommender systems evaluation. In: Proceedings of the SIGIR, pp. 2405–2414. ACM, Virtual Event Canada (2021)
Cañamares, R., Castells, P.: On target item sampling in offline recommender system evaluation. In: Proceedings of the RecSys, pp. 259–268 (2020)
Chang, H.S., Agarwal, N., McCallum, A.: To copy, or not to copy; that is a critical issue of the output Softmax layer in neural sequential recommenders. arXiv preprint arXiv:2310.14079. In: Proceedings of the WSDM, pp. 67–76 (2024)
Chang, H.S., McCallum, A.: Softmax bottleneck makes language models unable to represent multi-mode word distributions. In: Proceedings of the ACL, pp. 8048–8073. Association for Computational Linguistics, Dublin, Ireland (2022)
Chen, J., Wang, C., Wang, J., Yu, P.S.: Recommendation for repeat consumption from user implicit feedback. IEEE Trans. Knowl. Data Eng. 28(11), 3083–3097 (2016)
Cho, E., Myers, S.A., Leskovec, J.: Friendship and mobility: user movement in location-based social networks. In: Proceedings of the KDD, pp. 1082–1090 (2011)
Gao, L., et al.: PAL: program-aided language models. In: Proceedings of the 40th International Conference on Machine Learning, pp. 10764–10799. PMLR (2023)
Goldenberg, D., Levin, P.: Booking.com multi-destination trips dataset. In: Proceedings of the SIGIR, pp. 2457–2462 (2021)
Hidasi, B., Czapp, Á.T.: Widespread flaws in offline evaluation of recommender systems. In: Proceedings of the 17th ACM Conference on Recommender Systems, pp. 848–855. RecSys ’23, Association for Computing Machinery, New York, NY, USA (2023)
Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D.: Session-based recommendations with recurrent neural networks. In: Proceedings of the ICLR (2016)
Jannach, D., Kamehkhosh, I., Bonnin, G.: Music recommendations. In: Collaborative Recommendations, pp. 481–518. WORLD SCIENTIFIC (2018)
Jannach, D., Ludewig, M., Lerche, L.: Session-based item recommendation in e-commerce: on short-term intents, reminders, trends and discounts. User Model. User-Adap. Inter. 27(3), 351–392 (2017)
Kang, W.C., McAuley, J.: Self-attentive sequential recommendation, pp. 197–206
Klenitskiy, A., Vasilev, A.: Turning dross into gold loss: Is BERT4Rec really better than SASRec? In: Proceedings of the 17th ACM Conference on Recommender Systems, pp. 1120–1125. RecSys ’23, Association for Computing Machinery, New York, NY, USA (2023)
Krichene, W., Rendle, S.: On sampled metrics for item recommendation. Commun. ACM 65(7), 75–83 (2022)
Li, J., Sun, A., Ma, W., Sun, P., Zhang, M.: Recommender for its purpose: repeat and exploration in food delivery recommendations. In: Proceedings of the RecSys (2024)
Li, K., et al.: Communicative MARL-based relevance discerning network for repetition-aware recommendation. In: Proceedings of the ACM Web Conference 2023, pp. 1231–1239. ACM, Austin, TX, USA (2023)
Li, M., Vardasbi, A., Yates, A., De Rijke, M.: Repetition and exploration in sequential recommendation. In: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2532–2541. ACM, Taipei Taiwan (2023)
MacKay, D.: Information Theory, Inference and Learning Algorithms. Cambridge University Press (2003)
Petrov, A.V., Macdonald, C.: A systematic review and replicability study of BERT4Rec for sequential recommendation. In: Proceedings of the RecSys, pp. 436–447 (2022)
Petrov, A.V., Macdonald, C.: gSASRec: reducing overconfidence in sequential recommendation trained with negative sampling. In: Proceedings of the RecSys, pp. 116–128. ACM, Singapore (2023)
Quadrana, M., Cremonesi, P., Jannach, D.: Sequence-aware recommender systems. ACM CSUR 51(4), 66:1–66:36
Rappaz, J., McAuley, J., Aberer, K.: Recommendation on live-streaming platforms: dynamic availability and repeat consumption. In: Proceedings of the RecSys, pp. 390–399. ACM, Amsterdam Netherlands (2021)
Ravi, L., Vairavasundaram, S.: A collaborative location based travel recommendation system through enhanced rating prediction for the group of users. Comput. Intell. Neurosci. 2016, 1–28 (2016)
Ren, P., Chen, Z., Li, J., Ren, Z., Ma, J., de Rijke, M.: RepeatNet: a repeat aware neural recommendation machine for session-based recommendation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 4806–4813 (2019)
Schedl, M., Knees, P., McFee, B., Bogdanov, D.: Music recommendation systems: techniques, use cases, and challenges. In: Ricci, F., Rokach, L., Shapira, B. (eds.) Recommender Systems Handbook, pp. 927–971. Springer, US, New York, NY (2022)
Sinha, K.K., Suvvari, S.: Repetition dynamics-based deep learning model for next basket recommendation. SN Comput. Sci. 5(1), 97 (2023)
Sun, F., et al.: BERT4Rec: sequential recommendation with bidirectional encoder representations from transformer. In: Proceedings of the CIKM, pp. 1441–1450 (2018)
Tianchi: Ijcai-16 brick-and-mortar store recommendation dataset (2018). https://tianchi.aliyun.com/dataset/dataDetail?dataId=53
Tran, V.A., Salha-Galvan, G., Sguerra, B., Hennequin, R.: Transformers meet ACT-R: repeat-aware and sequential listening session recommendation. In: Proceedings of the RecSys (2024)
Yang, Z., Dai, Z., Salakhutdinov, R., Cohen, W.W.: Breaking the Softmax bottleneck: a high-rank RNN language model. arXiv (2018)
Yehudai, G., Kaplan, H., Ghandeharioun, A., Geva, M., Globerson, A.: When can transformers count to n? (2024)
Yue, Z., Wang, Y., He, Z., Zeng, H., Mcauley, J., Wang, D.: Linear recurrent units for sequential recommendation. In: Proceedings of the WSDM, pp. 930–938. WSDM ’24. Association for Computing Machinery, New York, NY, USA (2024)
Zhai, J., et al.: Revisiting neural retrieval on accelerators. In: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 5520–5531. ACM, Long Beach CA USA (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Petrov, A.V., Karra Taniskidou, E., Murphy, S. (2025). CountNet: Utilising Repetition Counts in Sequential Recommendation. In: Hauff, C., et al. Advances in Information Retrieval. ECIR 2025. Lecture Notes in Computer Science, vol 15574. Springer, Cham. https://doi.org/10.1007/978-3-031-88714-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-88714-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-88713-0
Online ISBN: 978-3-031-88714-7
eBook Packages: Computer ScienceComputer Science (R0)