Skip to main content
SpringerLink
Log in

A general tail item representation enhancement framework for sequential recommendation

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Recently advancements in deep learning models have significantly facilitated the development of sequential recommender systems (SRS). However, the current deep model structures are limited in their ability to learn high-quality embeddings with insufficient data. Meanwhile, highly skewed long-tail distribution is very common in recommender systems. Therefore, in this paper, we focus on enhancing the representation of tail items to improve sequential recommendation performance. Through empirical studies on benchmarks, we surprisingly observe that both the ranking performance and training procedure are greatly hindered by the poorly optimized tail item embeddings. To address this issue, we propose a sequential recommendation framework named TailRec that enables contextual information of tail item well-leveraged and greatly improves its corresponding representation. Given the characteristics of the sequential recommendation task, the surrounding interaction records of each tail item are regarded as contextual information without leveraging any additional side information. This approach allows for the mining of contextual information from cross-sequence behaviors to boost the performance of sequential recommendations. Such a light contextual filtering component is plug-and-play for a series of SRS models. To verify the effectiveness of the proposed TailRec, we conduct extensive experiments over several popular benchmark recommenders. The experimental results demonstrate that TailRec can greatly improve the recommendation results and speed up the training process. The codes of our methods have been available.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Wang S, Hu L, Wang Y, Cao L, Sheng Q Z, Orgun M A. Sequential recommender systems: challenges, progress and prospects. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 6332–6338

  2. Wu S, Tang Y, Zhu Y, Wang L, Xie X, Tan T. Session-based recommendation with graph neural networks. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence and 31st Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence. 2019, 43

  3. Xu E, Yu Z, Li N, Cui H, Yao L, Guo B. Quantifying predictability of sequential recommendation via logical constraints. Frontiers of Computer Science, 2023, 17(5): 175612

    Article  Google Scholar 

  4. Zhaok X, Liu H, Fan W, Liu H, Tang J, Wang C, Chen M, Zheng X, Liu X, Yang X. AutoEmb: automated embedding dimensionality search in streaming recommendations. In: Proceedings of 2021 IEEE International Conference on Data Mining. 2021, 896–905

  5. Cheng M, Liu Q, Liu Z, Li Z, Luo Y, Chen E. FormerTime: hierarchical multi-scale representations for multivariate time series classification. In: Proceedings of the ACM Web Conference. 2023, 1437–1445

  6. Cheng M, Liu Q, Liu Z, Zhang H, Zhang R, Chen E. TimeMAE: self-supervised representations of time series with decoupled masked autoencoders. 2023, arXiv preprint arXiv: 2303.00320

  7. Sun Y, Yuan F, Yang M, Wei G, Zhao Z, Liu D. A generic network compression framework for sequential recommender systems. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020, 1299–1308

  8. Chen L, Yuan F, Yang J, Ao X, Li C, Yang M. A user-adaptive layer selection framework for very deep sequential recommender models. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2021, 3984–3991

  9. Zhang S, Yao D, Zhao Z, Chua T S, Wu F. CauseRec: counterfactual user sequence synthesis for sequential recommendation. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, 367–377

  10. Yin J, Liu C, Wang W, Sun J, Hoi S C H. Learning transferrable parameters for long-tailed sequential user behavior modeling. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 359–367

  11. Kim Y, Kim K, Park C, Yu H. Sequential and diverse recommendation with long tail. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 2740–2746

  12. Fan Z, Liu Z, Zhang J, Xiong Y, Zheng L, Yu P S. Continuous-time sequential recommendation with temporal graph collaborative transformer. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021, 433–442

  13. Zipf G K. Human Behavior and the Principle of Least Effort: An Introduction to Human Ecology. Xue C F, trans. Shanghai: Shanghai People’s Publishing House, 2016

    Google Scholar 

  14. Yin H, Cui B, Li J, Yao J, Chen C. Challenging the long tail recommendation. Proceedings of the VLDB Endowment, 2012, 5(9): 896–907

    Article  Google Scholar 

  15. Liu Z, Cheng M, Li Z, Liu Q, Chen E. One person, one model-learning compound router for sequential recommendation. In: Proceedings of IEEE International Conference on Data Mining. 2022, 289–298

  16. Wei T, Feng F, Chen J, Wu Z, Yi J, He X. Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 1791–1800

  17. Liu S, Zheng Y. Long-tail session-based recommendation. In: Proceedings of the 14th ACM Conference on Recommender Systems. 2020, 509–514

  18. Jang S, Lee H, Cho H, Chung S. CITIES: contextual inference of tail-item embeddings for sequential recommendation. In: Proceedings of the 20th IEEE International Conference on Data Mining. 2020, 202–211

  19. Kang W C, McAuley J. Self-attentive sequential recommendation. In: Proceedings of 2018 IEEE International Conference on Data Mining. 2018, 197–206

  20. Rendle S, Freudenthaler C, Schmidt-Thieme L. Factorizing personalized Markov chains for next-basket recommendation. In: Proceedings of the 19th International Conference on World Wide Web. 2010, 811–820

  21. Hidasi B, Karatzoglou A, Baltrunas L, Tikk D. Session-based recommendations with recurrent neural networks. In: Proceedings of the 4th International Conference on Learning Representations. 2016

  22. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser L, Polosukhin I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000–6010

  23. Cheng M, Liu Z, Liu Q, Ge S, Chen E. Towards automatic discovering of deep hybrid network architecture for sequential recommendation. In: Proceedings of the ACM Web Conference 2022. 2022, 1923–1932

  24. Sun F, Liu J, Wu J, Pei C, Lin X, Ou W, Jiang P. BERT4Rec: sequential recommendation with bidirectional encoder representations from transformer. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2019, 1441–1450

  25. Cheng M, Yuan F, Liu Q, Ge S, Li Z, Yu R, Lian D, Yuan S, Chen E. Learning recommender systems with implicit feedback via soft target enhancement. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, 575–584

  26. Xue F, He X, Wang X, Xu J, Liu K, Hong R. Deep item-based collaborative filtering for top-N recommendation. ACM Transactions on Information Systems, 2019, 37(3): 33

    Article  Google Scholar 

  27. Cai Y, Cui Z, Wu S, Lei Z, Ma X. Represent items by items: An enhanced representation of the target item for recommendation. 2021, arXiv preprint arXiv: 2104.12483

  28. He K, Fan H, Wu Y, Xie S, Girshick R. Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 9726–9735

  29. Liu Q, Zeng Y, Mokhosi R, Zhang H. STAMP: short-term attention/memory priority model for session-based recommendation. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 1831–1839

  30. De Souza Pereira Moreira G, Rabhi S, Lee J M, Ak R, Oldridge E. Transformers4Rec: bridging the gap between NLP and sequential/session-based recommendation. In: Proceedings of the 15th ACM Conference on Recommender Systems. 2021, 143–153

  31. Tang J, Wang K. Personalized top-n sequential recommendation via convolutional sequence embedding. In: Proceedings of the 11th ACM International Conference on Web Search and Data Mining. 2018, 565–573

  32. Cheng M, Yuan F, Liu Q, Xin X, Chen E. Learning transferable user representations with sequential behaviors via contrastive pre-training. In: Proceedings of 2021 IEEE International Conference on Data Mining (ICDM). 2021, 51–60

  33. Li J, Ren P, Chen Z, Ren Z, Lian T, Ma J. Neural attentive session-based recommendation. In: Proceedings of 2017 ACM on Conference on Information and Knowledge Management. 2017, 1419–1428

  34. Yuan F, Karatzoglou A, Arapakis I, Jose J M, He X. A simple convolutional generative network for next item recommendation. In: Proceedings of the 12th ACM International Conference on Web Search and Data Mining. 2019, 582–590

  35. Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L. BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 2012, 452–461

  36. Zhang Y, Cheng D Z, Yao T, Yi X, Hong L, Chi E H. A model of two tales: dual transfer learning framework for improved long-tail item recommendation. In: Proceedings of the Web Conference 2021. 2021, 2220–2231

  37. Zhao Z, Chen J, Zhou S, He X, Cao X, Zhang F, Wu W. Popularity bias is not always evil: disentangling benign and harmful bias for recommendation. IEEE Transactions on Knowledge and Data Engineering, 2022, doi: https://doi.org/10.1109/TKDE.2022.3218994

  38. He X, Chua T S. Neural factorization machines for sparse predictive analytics. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2017, 355–364

  39. Pi Q, Zhou G, Zhang Y, Wang Z, Ren L, Fan Y, Zhu X, Gai K. Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020, 2685–2692

  40. He R, Fang C, Wang Z, McAuley J. Vista: a visually, socially, and temporally-aware model for artistic recommendation. In: Proceedings of the 10th ACM Conference on Recommender Systems. 2016, 309–316

  41. Ying H, Zhuang F, Zhang F, Liu Y, Xu G, Xie X, Xiong H, Wu J. Sequential recommender system based on hierarchical attention network. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018, 3926–3932

  42. Gu Y, Lei T, Barzilay R, Jaakkola T. Learning to refine text based recommendations. In: Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing. 2016, 2103–2108

  43. Tan Y K, Xu X, Liu Y. Improved recurrent neural networks for session-based recommendations. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. 2016, 17–22

  44. Huang J, Zhao W X, Dou H, Wen J R, Chang E Y. Improving sequential recommendation with knowledge-enhanced memory networks. In: Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 2018, 505–514

  45. Wang J, Yuan F, Chen J, Wu Q, Yang M, Sun Y, Zhang G. StackRec: efficient training of very deep sequential recommender models by iterative stacking. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, 357–366

  46. Liang Y, Song Q, Zhao Z, Zhou H, Gong M. BA-GNN: Behavior-aware graph neural network for session-based recommendation. Frontiers of Computer Science, 2023, 17(6): 176613

    Article  Google Scholar 

  47. Brynjolfsson E, Hu Y J, Smith M D. From niches to riches: anatomy of the long tail. Sloan Management Review, 2006, 47(4): 67–71

    Google Scholar 

  48. Liang D, Charlin L, Blei D M. Causal inference for recommendation. In: Proceedings of Causation: Foundation to Application, Workshop at UAI. 2016

  49. Abdollahpouri H, Burke R, Mobasher B. Controlling popularity bias in learning-to-rank recommendation. In: Proceedings of the 11th ACM Conference on Recommender Systems. 2017, 42–46

  50. Adomavicius G, Kwon Y. Improving aggregate recommendation diversity using ranking-based techniques. IEEE Transactions on Knowledge and Data Engineering, 2012, 24(5): 896–911

    Article  Google Scholar 

  51. Bai B, Fan Y, Tan W, Zhang J. DLTSR: a deep learning framework for recommendations of long-tail web services. IEEE Transactions on Services Computing, 2020, 13(1): 73–85

    Article  Google Scholar 

  52. Li J, Lu K, Huang Z, Shen H T. Two birds one stone: on both cold-start and long-tail recommendation. In: Proceedings of the 25th ACM International Conference on Multimedia. 2017, 898–906

Download references

Acknowledgements

This research was supported by grant from the National Key R&D Program of China (No. 2021YFF0901003). The authors also appreciate the partial support from Hefei AI Computing Center (Project Team).

Author information

Authors and Affiliations

  1. Anhui Province Key Laboratory of Big Data Analysis and Application, University of Science and Technology of China & State Key Laboratory of Cognitive Intelligence, Hefei, 230026, China

    Mingyue Cheng, Qi Liu, Wenyu Zhang, Zhiding Liu & Enhong Chen

  2. College of Management and Economics, Tianjin University, Tianjin, 300072, China

    Hongke Zhao

Authors
  1. Mingyue Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenyu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhiding Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hongke Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Enhong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qi Liu.

Ethics declarations

Competing interests The authors declare that they have no competing interests or financial conflicts to disclose.

Additional information

Mingyue Cheng is currently pursuing PhD degree with the Anhui Province Key Laboratory of Big Data Analysis and Application (BDAA), School of Data Science, University of Science and Technology of China (USTC), China. Before that, he received BArts degree from Hefei University of Technology (HFUT), China. His general research interests span machine learning methods and their applications, especially focusing on sequence data mining and prediction, such sequential behaviors analysis, time series analysis. He has published more than 10 papers in referred journals and conference proceedings, e.g., the TKDE, WWW, SIGIR, ICDM. He also serves on the program committees of conferences, including SIGKDD and SIGIR.

Qi Liu received the PhD degree from University of Science and Technology of China (USTC), China in 2013. He is currently a professor in the School of Computer Science and Technology at USTC. His general area of research is data mining and knowledge discovery. He has published prolifically in refereed journals and conference proceedings (e.g., TKDE, TOIS, KDD). He is an Associate Editor of IEEE TBD and Neurocomputing. He was the recipient of KDD’18 Best Student Paper Award and ICDM’11 Best Research Paper Award. He is a member of the Alibaba DAMO Academy Young Fellow. He was also the recipient of China Outstanding Youth Science Foundation in 2019.

Wenyu Zhang is currently working toward the PhD degree in the School of Computer Science and Technology at University of Science and Technologyof China (USTC), China. He received his bachelor degree in communication engineering from Anhui Agricultural University (AHAU), China in 2021. His main research interests include recommender systems, and computer vision. He has published papers in referred conference proceedings, such as ACM MM’2021 and BMVC’2021.

Zhiding Liu received his BE degree in computer science from University of Science and Technology of China (USTC), China in 2021. He is currently working toward the ME degree in the School of Computer Science and Technology at University of Science and Technology of China (USTC), China. His main research interests include data mining, recommender systems, and time series representation learning. He has published papers in referred conference proceedings, such as WWW’2022 and ICDM’2022.

Hongke Zhao received the PhD degree from the University of Science and Technology of China (USTC), China. He is an associate professor with the College of Management and Economics, Tianjin University, China. His research interest includes data mining, data-driven management, knowledge and behavior computing. He has published more than 70 papers in refereed journals and conference proceedings, such as INFORMS Journal on Computing, IEEE Transactions on Knowledge and Data Engineering, IEEE Transactions on Evolutionary Computation, ACM Transactions on Information Systems, ACM SIGKDD, ACM WSDM, ACM SIGIR, IJCAI, AAAI, and IEEE ICDM. He was the recipient of Distinguished Dissertation Award Nomination of CAAI (2019), the Best Student Paper Award of CCML-2019.

Enhong Chen received the PhD degree from the University of Science and Technology of China (USTC), China. He is a professor and vice dean of the School of Computer Science, USTC, China. His general area of research includes data mining and machine learning, social network analysis, and recommender systems. He has published more than 100 papers in refereed conferences and journals, including the IEEE Transactions on Knowledge and Data Engineering, the IEEE Transactions on Mobile Computing, the IEEE Transactions on Industrial Electronics, the ACM Transactions on Knowledge Discovery from Data, ACM SIGKDD, IEEE ICDM, and NIPS. He was on program committees of numerous conferences including SIGKDD, ICDM, and SDM. He received the Best Application Paper Award on KDD’2008, the Best Research Paper Award on ICDM’ 2011, and the Best of SDM’2015. His research is supported by the National Science Foundation for Distinguished Young Scholars of China.

Electronic Supplementary Material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, M., Liu, Q., Zhang, W. et al. A general tail item representation enhancement framework for sequential recommendation. Front. Comput. Sci. 18, 186333 (2024). https://doi.org/10.1007/s11704-023-3112-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-023-3112-y

Keywords

Search

Navigation