Abstract
The core of sequential recommendations is to model users’ dynamic preferences from their sequential historical behaviors. Bidirectional representation models can make better sequential recommendations because each item in user’s historical behaviors fuses information from both left and right sides. Despite their effectiveness, we argue that such bidirectional models are sub-optimal due to the limitations including: a) items with the same timestamp interactions have adverse effect on user modeling; b) the random masking process often produces noises. To address these limitations, we propose Multi-pair Contrastive Learning based on same-timestamp data augmentation for Sequential Recommendation (MCL4SR). Specifically, we firstly modify the masking strategies of BERT encoder. Then we propose a multi-pair contrastive learning framework by exploring data augmentation of the same timestamp interactions. During the training and testing process, we design three types of samples so as to imitate human learning. Extensive experiments on two benchmark datasets show that our model outperforms state-of-the-art sequential models.
This work was supported by Shandong Provincial Natural Science Foundation, China (ZR2020MF147, ZR2021MF017).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT 2019, Minneapolis, 2–7 June 2019, vol. 1, pp. 4171–4186 (2019)
Sun, F., et al.: Bert4rec: sequential recommendation with bidirectional encoder representations from transformer. In: CIKM 2019, Beijing, 3–7 November 2019, pp. 1441–1450 (2019)
Du, H., et al.: Contrastive learning with bidirectional transformers for sequential recommendation. In: CIKM, Atlanta, 17–21 October 2022, pp. 396–405 (2022)
Wettig, A., Gao, T., Zhong, Z., Chen, D.: Should you mask 15% in masked language modeling? In: EACL 2023, Dubrovnik, 2–6 May 2023, pp. 2977–2992 (2023)
Zhou, K., et al.: S3-rec: self-supervised learning for sequential recommendation with mutual information maximization. In: CIKM 2020, Virtual Event, 19–23 October 2020, pp. 1893–1902 (2020)
Xie, X., et al.: Contrastive learning for sequential recommendation. In: ICDE 2022, Kuala Lumpur, 9–12 May 2022, pp. 1259–1273 (2022)
Liu, Z., Chen, Y., Li, J., Yu, P.S., McAuley, J.J., Xiong, C.: Contrastive self-supervised sequential recommendation with robust augmentation. arXiv preprint arXiv:2108.06479 (2021)
Qiu, R., Huang, Z., Yin, H., Wang, Z.: Contrastive learning for representation degeneration problem in sequential recommendation. In: WSDM 2022 Virtual Event/Tempe, 21-25 February 2022, pp. 813–823 (2022)
Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Shani, G., Heckerman, D., Brafman, R.I.: An MDP-based recommender system. J. Mach. Learn. Res. 6, 1265–1295 (2005)
Rendle, S., Freudenthaler, C., Schmidt-Thieme, L.: Factorizing personalized Markov chains for next-basket recommendation. In: WWW 2010, Raleigh, 26–30 April 2010, pp. 811–820 (2010)
He, R., McAuley, J.J.: Fusing similarity models with Markov chains for sparse sequential recommendation. In: ICDM 2016, 12–15 December 2016, Barcelona, pp. 191–200 (2016)
Kang, W., McAuley, J.J.: Self-attentive sequential recommendation. In: ICDM 2018, Singapore, 17–20 November 2018, pp. 197–206 (2018)
Xu, C., et al.: Long- and short-term self-attention network for sequential recommendation. Neurocomputing 423, 580–589 (2021)
Wu, S., Tang, Y., Zhu, Y., Wang, L., Xie, X., Tan, T.: Session-based recommendation with graph neural networks. In: EAAI 2019, Honolulu, 27 January–1 February 2019, pp. 346–353 (2019)
Xu, C., et al.: Graph contextualized self-attention network for session-based recommendation. In: IJCAI 2019, Macao, 10–16 August 2019, pp. 3940–3946 (2019)
Zhang, T., et al.: Feature-level deeper self-attention network for sequential recommendation. In: IJCAI 2019, Macao, 10–16 August 2019, pp. 4320–4326 (2019)
McAuley, J.J., Targett, C., Shi, Q., van den Hengel, A.: Image-based recommendations on styles and substitutes. In: SIGIR 2015, Santiago, 9–13 August 2015, pp. 43–52 (2015)
Harper, F.M., Konstan, J.A.: The movielens datasets: history and context. ACM Trans. Interact. Intell. Syst. 5(4), 19:1–19:19 (2016)
Krichene, W., Rendle, S.: On sampled metrics for item recommendation. Commun. ACM 65(7), 75–83 (2022)
Zhao, W.X., et al.: Recbole: towards a unified, comprehensive and efficient framework for recommendation algorithms. In: CIKM 2021, Virtual Event, Queensland, 1–5 November 2021, pp. 4653–4664 (2021)
Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: NeurIPS 2019, 8–14 December 2019, Vancouver, pp. 8024–8035 (2019)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR 2015, San Diego, 7–9, May 2015, Conference Track Proceedings (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zheng, S., Wang, S., Zhang, L., Zhang, Y., Sun, F. (2024). Multi-pair Contrastive Learning Based on Same-Timestamp Data Augmentation for Sequential Recommendation. In: Song, X., Feng, R., Chen, Y., Li, J., Min, G. (eds) Web and Big Data. APWeb-WAIM 2023. Lecture Notes in Computer Science, vol 14333. Springer, Singapore. https://doi.org/10.1007/978-981-97-2387-4_12
Download citation
DOI: https://doi.org/10.1007/978-981-97-2387-4_12
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2386-7
Online ISBN: 978-981-97-2387-4
eBook Packages: Computer ScienceComputer Science (R0)