Skip to main content

Context Preserving Data Augmentation for Sequential Recommendation

  • Conference paper
  • First Online:
Services Computing – SCC 2022 (SCC 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13738))

Included in the following conference series:

  • 317 Accesses

Abstract

Item representation learning is a fundamental task in Sequential Recommendation (SR). Effective representations are crucial for SR because they enable recommender systems to learn relevant relationships between items. SR researchers rely on User Historical Interactions (UHI) for effective item representations. While it is well understood that UHI inherently suffers from data sparsity, which weakens item relation signals, seldom considered is the fact the interaction between users and items is mediated by an underlying candidate generation process susceptible to bias, noise and error. These limitations further distort the item relationships and limit the learning of superior item representations. In this work, we seek to amplify weak item relation signals in UHI by augmenting each input sequence with a set of permutations that preserve both the local and global context. We employ a multi-layer bi-directional transformer encoder to learn superior contextualized item representations from the augmented data. Extensive experiments on benchmark datasets for next-item recommendations demonstrate that our proposed SR model can recover item relational dynamics distorted during the candidate generation process. In addition, our approach leads to learning superior item representations for many next-item state-of-the-art models employing RNNs and self-attention networks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The 3-D representation of UHI is created by concatenating 2-D representation of all the items in the UHI in their original order.

References

  1. Amankwata, B., Fletcher, K.K.: Contexts embedding for sequential service recommendation. In: 2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC), pp. 1087–1092. IEEE (2022)

    Google Scholar 

  2. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473 (2015)

    Google Scholar 

  3. Blumberg, O.: Permutations with interval restrictions. Ph.D. thesis, PhD thesis, Stanford University (2012)

    Google Scholar 

  4. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.E.: A simple framework for contrastive learning of visual representations. ArXiv abs/2002.05709 (2020)

    Google Scholar 

  5. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: NAACL (2019)

    Google Scholar 

  6. Fletcher, K.: Regularizing matrix factorization with implicit user preference embeddings for web API recommendation. In: 2019 IEEE International Conference on Services Computing (SCC), pp. 1–8. IEEE (2019)

    Google Scholar 

  7. Fletcher, K.K.: A quality-aware web API recommender system for mashup development. In: Ferreira, J.E., Musaev, A., Zhang, L.-J. (eds.) SCC 2019. LNCS, vol. 11515, pp. 1–15. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23554-3_1

    Chapter  Google Scholar 

  8. Grbovic, M., et al.: E-commerce in your inbox: product recommendations at scale. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2015)

    Google Scholar 

  9. He, R., McAuley, J.: Fusing similarity models with markov chains for sparse sequential recommendation. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 191–200 (2016)

    Google Scholar 

  10. Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D.: Session-based recommendations with recurrent neural networks. CoRR abs/1511.06939 (2016)

    Google Scholar 

  11. Hidasi, B., Quadrana, M., Karatzoglou, A., Tikk, D.: Parallel recurrent neural network architectures for feature-rich session-based recommendations. In: Proceedings of the 10th ACM Conference on Recommender Systems (2016)

    Google Scholar 

  12. Kang, W.C., McAuley, J.: Self-attentive sequential recommendation. In: 2018 IEEE International Conference on Data Mining (ICDM), pp. 197–206 (2018)

    Google Scholar 

  13. Kwapong, B.A., Anarfi, R., Fletcher, K.K.: Personalized service recommendation based on user dynamic preferences. In: Ferreira, J.E., Musaev, A., Zhang, L.-J. (eds.) SCC 2019. LNCS, vol. 11515, pp. 77–91. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23554-3_6

    Chapter  Google Scholar 

  14. Lastfm: Last.fm dataset - 1k users. http://ocelma.net/MusicRecommendationDataset/lastfm-1K.html. Accessed 30 Dec 2020

  15. Liu, Y., et al.: Roberta: a robustly optimized Bert pretraining approach. ArXiv abs/1907.11692 (2019)

    Google Scholar 

  16. Liu, Z., Chen, Y.G., Li, J., Yu, P.S., McAuley, J., Xiong, C.: Contrastive self-supervised sequential recommendation with robust augmentation. ArXiv abs/2108.06479 (2021)

    Google Scholar 

  17. McFee, B., Lanckriet, G.: The natural language of playlists. In: ISMIR (2011)

    Google Scholar 

  18. Miaschi, A., Dell’Orletta, F.: Contextual and non-contextual word embeddings: an in-depth linguistic investigation. In: REPL4NLP (2020)

    Google Scholar 

  19. Mikolov, T., Chen, K., Corrado, G.S., Dean, J.: Efficient estimation of word representations in vector space. In: ICLR (2013)

    Google Scholar 

  20. MovieLense: Movielens 25m dataset. https://grouplens.org/datasets/movielens/25m/. Accessed 30 Dec 2020

  21. Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: EMNLP (2014)

    Google Scholar 

  22. Rendle, S., Freudenthaler, C., Schmidt-Thieme, L.: Factorizing personalized markov chains for next-basket recommendation. In: WWW 2010 (2010)

    Google Scholar 

  23. Sahlgren, M., Holst, A., Kanerva, P.: Permutations as a means to encode order in word space (2008)

    Google Scholar 

  24. Song, Y., Lee, J.: Augmenting recurrent neural networks with high-order user-contextual preference for session-based recommendation. ArXiv abs/1805.02983 (2018)

    Google Scholar 

  25. Song, Y., Wang, J., Liang, Z., Liu, Z., Jiang, T.: Utilizing Bert intermediate layers for aspect based sentiment analysis and natural language inference. ArXiv abs/2002.04815 (2020)

    Google Scholar 

  26. Sun, F., et al.: Bert4rec: sequential recommendation with bidirectional encoder representations from transformer. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management (2019)

    Google Scholar 

  27. Tang, J., Wang, K.: Personalized top-n sequential recommendation via convolutional sequence embedding. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (2018)

    Google Scholar 

  28. Tian, Y., Sun, C., Poole, B., Krishnan, D., Schmid, C., Isola, P.: What makes for good views for contrastive learning. ArXiv abs/2005.10243 (2020)

    Google Scholar 

  29. Turrin, R., Quadrana, M., Condorelli, A., Pagano, R., Cremonesi, P.: 30music listening and playlists dataset. In: RecSys Posters (2015)

    Google Scholar 

  30. Vasile, F., Smirnova, E., Conneau, A.: Meta-prod2vec: Product embeddings using side-information for recommendation. In: Proceedings of the 10th ACM Conference on Recommender Systems (2016)

    Google Scholar 

  31. Vaswani, A.,et al.: Attention is all you need. In: NIPS (2017)

    Google Scholar 

  32. Wu, J., et al.: Self-supervised graph learning for recommendation. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (2021)

    Google Scholar 

  33. Wu, J., Cai, R., Wang, H.: Déjà vu: a contextualized temporal attention mechanism for sequential recommendation. In: Proceedings of The Web Conference 2020 (2020)

    Google Scholar 

  34. Wu, S., Tang, Y., Zhu, Y., Wang, L., Xie, X., Tan, T.: Session-based recommendation with graph neural networks. In: AAAI (2019)

    Google Scholar 

  35. Xie, X., Sun, F., Liu, Z., Gao, J., Ding, B., Cui, B.: Contrastive pre-training for sequential recommendation. ArXiv abs/2010.14395 (2020)

    Google Scholar 

  36. Xu, B., Wang, N., Chen, T., Li, M.: Empirical evaluation of rectified activations in convolutional network. ArXiv abs/1505.00853 (2015)

    Google Scholar 

  37. Yang, D., Zhang, D., Zheng, V., Yu, Z.: Modeling user activity preference by leveraging user spatial temporal characteristics in LBSNs. IEEE Trans. Syst. Man, Cybern. Syst. 45, 129–142 (2015)

    Article  Google Scholar 

  38. Yang, J., Zhao, H.: Deepening hidden representations from pre-trained language models for natural language understanding. ArXiv abs/1911.01940 (2019)

    Google Scholar 

  39. Yelp: The dataset. https://www.yelp.com/dataset. Accessed 30 Dec 2020

  40. Zangerle, E., Pichl, M., Gassler, W., Specht, G.: nowplaying music dataset: Extracting listening behavior from twitter. In: WISMM 2014 (2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Kenneth K. Fletcher .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Amankwata, B., Fletcher, K.K. (2022). Context Preserving Data Augmentation for Sequential Recommendation. In: Qingyang, W., Zhang, LJ. (eds) Services Computing – SCC 2022. SCC 2022. Lecture Notes in Computer Science, vol 13738. Springer, Cham. https://doi.org/10.1007/978-3-031-23515-3_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-23515-3_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-23514-6

  • Online ISBN: 978-3-031-23515-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics