Abstract
Item representation learning is a fundamental task in Sequential Recommendation (SR). Effective representations are crucial for SR because they enable recommender systems to learn relevant relationships between items. SR researchers rely on User Historical Interactions (UHI) for effective item representations. While it is well understood that UHI inherently suffers from data sparsity, which weakens item relation signals, seldom considered is the fact the interaction between users and items is mediated by an underlying candidate generation process susceptible to bias, noise and error. These limitations further distort the item relationships and limit the learning of superior item representations. In this work, we seek to amplify weak item relation signals in UHI by augmenting each input sequence with a set of permutations that preserve both the local and global context. We employ a multi-layer bi-directional transformer encoder to learn superior contextualized item representations from the augmented data. Extensive experiments on benchmark datasets for next-item recommendations demonstrate that our proposed SR model can recover item relational dynamics distorted during the candidate generation process. In addition, our approach leads to learning superior item representations for many next-item state-of-the-art models employing RNNs and self-attention networks.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The 3-D representation of UHI is created by concatenating 2-D representation of all the items in the UHI in their original order.
References
Amankwata, B., Fletcher, K.K.: Contexts embedding for sequential service recommendation. In: 2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC), pp. 1087–1092. IEEE (2022)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473 (2015)
Blumberg, O.: Permutations with interval restrictions. Ph.D. thesis, PhD thesis, Stanford University (2012)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.E.: A simple framework for contrastive learning of visual representations. ArXiv abs/2002.05709 (2020)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: NAACL (2019)
Fletcher, K.: Regularizing matrix factorization with implicit user preference embeddings for web API recommendation. In: 2019 IEEE International Conference on Services Computing (SCC), pp. 1–8. IEEE (2019)
Fletcher, K.K.: A quality-aware web API recommender system for mashup development. In: Ferreira, J.E., Musaev, A., Zhang, L.-J. (eds.) SCC 2019. LNCS, vol. 11515, pp. 1–15. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23554-3_1
Grbovic, M., et al.: E-commerce in your inbox: product recommendations at scale. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2015)
He, R., McAuley, J.: Fusing similarity models with markov chains for sparse sequential recommendation. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 191–200 (2016)
Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D.: Session-based recommendations with recurrent neural networks. CoRR abs/1511.06939 (2016)
Hidasi, B., Quadrana, M., Karatzoglou, A., Tikk, D.: Parallel recurrent neural network architectures for feature-rich session-based recommendations. In: Proceedings of the 10th ACM Conference on Recommender Systems (2016)
Kang, W.C., McAuley, J.: Self-attentive sequential recommendation. In: 2018 IEEE International Conference on Data Mining (ICDM), pp. 197–206 (2018)
Kwapong, B.A., Anarfi, R., Fletcher, K.K.: Personalized service recommendation based on user dynamic preferences. In: Ferreira, J.E., Musaev, A., Zhang, L.-J. (eds.) SCC 2019. LNCS, vol. 11515, pp. 77–91. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23554-3_6
Lastfm: Last.fm dataset - 1k users. http://ocelma.net/MusicRecommendationDataset/lastfm-1K.html. Accessed 30 Dec 2020
Liu, Y., et al.: Roberta: a robustly optimized Bert pretraining approach. ArXiv abs/1907.11692 (2019)
Liu, Z., Chen, Y.G., Li, J., Yu, P.S., McAuley, J., Xiong, C.: Contrastive self-supervised sequential recommendation with robust augmentation. ArXiv abs/2108.06479 (2021)
McFee, B., Lanckriet, G.: The natural language of playlists. In: ISMIR (2011)
Miaschi, A., Dell’Orletta, F.: Contextual and non-contextual word embeddings: an in-depth linguistic investigation. In: REPL4NLP (2020)
Mikolov, T., Chen, K., Corrado, G.S., Dean, J.: Efficient estimation of word representations in vector space. In: ICLR (2013)
MovieLense: Movielens 25m dataset. https://grouplens.org/datasets/movielens/25m/. Accessed 30 Dec 2020
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: EMNLP (2014)
Rendle, S., Freudenthaler, C., Schmidt-Thieme, L.: Factorizing personalized markov chains for next-basket recommendation. In: WWW 2010 (2010)
Sahlgren, M., Holst, A., Kanerva, P.: Permutations as a means to encode order in word space (2008)
Song, Y., Lee, J.: Augmenting recurrent neural networks with high-order user-contextual preference for session-based recommendation. ArXiv abs/1805.02983 (2018)
Song, Y., Wang, J., Liang, Z., Liu, Z., Jiang, T.: Utilizing Bert intermediate layers for aspect based sentiment analysis and natural language inference. ArXiv abs/2002.04815 (2020)
Sun, F., et al.: Bert4rec: sequential recommendation with bidirectional encoder representations from transformer. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management (2019)
Tang, J., Wang, K.: Personalized top-n sequential recommendation via convolutional sequence embedding. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (2018)
Tian, Y., Sun, C., Poole, B., Krishnan, D., Schmid, C., Isola, P.: What makes for good views for contrastive learning. ArXiv abs/2005.10243 (2020)
Turrin, R., Quadrana, M., Condorelli, A., Pagano, R., Cremonesi, P.: 30music listening and playlists dataset. In: RecSys Posters (2015)
Vasile, F., Smirnova, E., Conneau, A.: Meta-prod2vec: Product embeddings using side-information for recommendation. In: Proceedings of the 10th ACM Conference on Recommender Systems (2016)
Vaswani, A.,et al.: Attention is all you need. In: NIPS (2017)
Wu, J., et al.: Self-supervised graph learning for recommendation. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (2021)
Wu, J., Cai, R., Wang, H.: Déjà vu: a contextualized temporal attention mechanism for sequential recommendation. In: Proceedings of The Web Conference 2020 (2020)
Wu, S., Tang, Y., Zhu, Y., Wang, L., Xie, X., Tan, T.: Session-based recommendation with graph neural networks. In: AAAI (2019)
Xie, X., Sun, F., Liu, Z., Gao, J., Ding, B., Cui, B.: Contrastive pre-training for sequential recommendation. ArXiv abs/2010.14395 (2020)
Xu, B., Wang, N., Chen, T., Li, M.: Empirical evaluation of rectified activations in convolutional network. ArXiv abs/1505.00853 (2015)
Yang, D., Zhang, D., Zheng, V., Yu, Z.: Modeling user activity preference by leveraging user spatial temporal characteristics in LBSNs. IEEE Trans. Syst. Man, Cybern. Syst. 45, 129–142 (2015)
Yang, J., Zhao, H.: Deepening hidden representations from pre-trained language models for natural language understanding. ArXiv abs/1911.01940 (2019)
Yelp: The dataset. https://www.yelp.com/dataset. Accessed 30 Dec 2020
Zangerle, E., Pichl, M., Gassler, W., Specht, G.: nowplaying music dataset: Extracting listening behavior from twitter. In: WISMM 2014 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Amankwata, B., Fletcher, K.K. (2022). Context Preserving Data Augmentation for Sequential Recommendation. In: Qingyang, W., Zhang, LJ. (eds) Services Computing – SCC 2022. SCC 2022. Lecture Notes in Computer Science, vol 13738. Springer, Cham. https://doi.org/10.1007/978-3-031-23515-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-23515-3_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-23514-6
Online ISBN: 978-3-031-23515-3
eBook Packages: Computer ScienceComputer Science (R0)