Skip to main content
SpringerLink
Log in

Transformer-Based Rating-Aware Sequential Recommendation

  • Conference paper
  • First Online:
Algorithms and Architectures for Parallel Processing (ICA3PP 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 13155))

Abstract

As a significant application of Big Data, recommender system can effectively solve information overload. The user's behavior sequence forms massive data and has excellent mining value. Sequential recommendation is to extract user's features in massive sequential data and predict the next interaction based on the user's recent temporal behavior. Currently, recurrent neural networks (RNN) and Graph Neural Networks (GNN) take on the role of item embedding in sequential recommendation and have shown adequate performance. However, such RNN based model and GNN based model cannot deeply mine the complex behavior sequence and neglect user preference like rating information. Inspired by the popular Transformer, we adopt the Transformer encoder layer to process sequence and represent item embedding by multi-head attention. Meanwhile, rating information is integrated into weight calculation when we represent the user preference with self-attention. Weight with rating not only retains the structural information of sequence but also combines the user's preferences. What's more, we consider global and local preferences to formulate hybrid performance and make recommendations in Top-N. For persuasiveness, we conduct experiments on large real-world datasets, and our model performs better in most cases on two datasets compared to state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Wu, L., Yu, H.-F., Rao, N., Sharpnack, J., Hsieh, C.: Graph DNA: Deep neighborhood aware graph encoding for collaborative filtering. In: The 23rd International Conference on Artificial Intelligence and Statistics, 2020, Online, pp. 776–787. PMLR (2019)

    Google Scholar 

  2. Wang, B., Cai, W.: Knowledge-enhanced graph neural networks for sequential recommendation. Inf. 11 (2020)

    Google Scholar 

  3. Rendle, S., Freudenthaler, C., Schmidt-Thieme, L.: Factorizing personalized Markov chains for next-basket recommendation. In: Proceedings of the 19th International Conference on World Wide Web, WWW 2010, pp. 811–820. ACM (2010)

    Google Scholar 

  4. Cai, C., He, R., McAuley, J.: SPMC: Socially-aware personalized Markov chains for sparse sequential recommendation. In: IJCAI International Joint Conference on Artificial Intelligence, pp. 1476–1482. ijcai.org (2017)

    Google Scholar 

  5. Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D.: Session-based recommendations with recurrent neural networks. In: 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings (2016)

    Google Scholar 

  6. Li, J., Ren, P., Chen, Z., Ren, Z., Lian, T., Ma, J.: Neural attentive session-based recommendation. In: International Conference on Information and Knowledge Management, Proceedings, pp. 1419–1428. ACM (2017)

    Google Scholar 

  7. Wu, S., Tang, Y., Zhu, Y., Wang, L., Xie, X., Tan, T.: Session-based recommendation with graph neural networks. In: 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, pp. 346–353. Press (2019)

    Google Scholar 

  8. Kang, W.C., McAuley, J.: Self-attentive sequential recommendation. In: Proceedings - IEEE International Conference on Data Mining, ICDM, pp. 197–206. IEEE (2018)

    Google Scholar 

  9. Yu, F., Zhu, Y., Liu, Q., Wu, S., Wang, L., Tan, T.: TAGNN: target attentive graph neural networks for session-based recommendation. In: SIGIR 2020 - Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1921–1924 (2020)

    Google Scholar 

  10. Pan, Z., Cai, F., Chen, W., Chen, H., De Rijke, M.: star graph neural networks for session-based recommendation. In: International Conference on Information and Knowledge Management, Proceedings, pp. 1195–1204 (2020)

    Google Scholar 

  11. Liu, Q., Mokhosi, R., Zeng, Y., Zhang, H.: STAMP: short-term attention/memory priority model for session-based recommendation. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1831–1839. ACM (2018)

    Google Scholar 

  12. Ren, R., Liu, Z., Li, Y., Zhao, W.X., Wang, H., Ding, B., Wen, J.R.: Sequential Recommendation with Self-Attentive Multi-Adversarial Network. In: SIGIR 2020 - Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 89–98. ACM (2020)

    Google Scholar 

  13. Wu, L., Li, S., Hsieh, C.J., Sharpnack, J.: SSE-PT: sequential recommendation via personalized transformer. In: RecSys 2020 - 14th ACM Conference on Recommender Systems, pp. 328–337 (2020)

    Google Scholar 

  14. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5999–6009 (2017)

    Google Scholar 

  15. Oliveira, J., Nogueira, M., Ramos, C., Renna, F., Ferreira, C., Coimbra, M.: Using soft attention mechanisms to classify heart sounds. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, pp. 6669–6672. IEEE (2019)

    Google Scholar 

  16. Da Conceição Moreira, P.S., Tsunoda, D.F.: LAST.FM songs database: a database for musical genre classification. In: IC3K 2018 - Proceedings of the 10th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, pp. 253–260. SciTePress (2018)

    Google Scholar 

  17. Xu, C., Feng, J., Zhao, P., Zhuang, F., Wang, D., Liu, Y.: Long- and short-term self-attention network for sequential recommendation. Neurocomputing 423, 580–589 (2021)

    Article  Google Scholar 

  18. Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web, WWW 2001, pp. 285–295. ACM (2001)

    Google Scholar 

  19. Zhao, W.X., Mu, S., Hou, Y.: RecBole: towards a unified, comprehensive and efficient framework for recommendation algorithms. In: The 30th International Conference on Information and Knowledge Management, Virtual Event, 2021. pp. 4653–4664. ACM (2021)

    Google Scholar 

  20. Lonjarret, C., Auburtin, R., Robardet, C., Plantevit, M.: Sequential recommendation with metric models based on frequent sequences. Data Min. Knowl. Disc. 35(3), 1087–1133 (2021). https://doi.org/10.1007/s10618-021-00744-w

    Article  MathSciNet  Google Scholar 

  21. Siy, P.W., et al.: Matrix factorization techniques for analysis of imaging mass spectrometry data. In: 8th IEEE International Conference on BioInformatics and BioEngineering, BIBE 2008, pp. 1–6. IEEE (2008)

    Google Scholar 

  22. Zhang, X.Q., Liu, X.X., Guo, J., Liu, B.Y., Gan, D.G.: A matrix factorization based recommendation algorithm for science and technology resource exploitation. In: Proceedings of IEEE/ACS International Conference on Computer Systems and Applications, AICCSA, pp. 1–6. IEEE (2020)

    Google Scholar 

  23. He, X., Liao, L., Zhang, H., Nie, L., Hu, X., Chua, T.S.: Neural collaborative filtering. In: 26th International World Wide Web Conference, WWW 2017, pp. 173–182. ACM (2017)

    Google Scholar 

  24. Meng, S., Gao, Z., Li, Q., Wang, H., Dai, H.N., Qi, L.: Security-Driven hybrid collaborative recommendation method for cloud-based iot services. Comput. Secur. 97, 101950 (2020)

    Google Scholar 

  25. Wang, P., Guo, J., Lan, Y.: Learning hierarchical representation model for next basket recommendation. In: SIGIR 2015 - Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 403–412. ACM (2015)

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Key R&D Program (2020YFB1804604), the 2020 Industrial Internet Innovation and Development Project from Ministry of Industry and Information Technology ,Jiangsu Province Modern Education Technology Research Project (84365); Scientific research project of Nanjing Vocational University of Industry Technology(2020SKYJ03).

Author information

Authors and Affiliations

  1. Nanjing University of Science and Technology, Nanjing, China

    Yang Li, Qianmu Li & Shunmei Meng

  2. School of Social Science, Nanjing Vocational University of Industry Technology, Nanjing, China

    Jun Hou

  3. Intelligent Manufacturing Department, Wuyi University, Jiangmen, China

    Jun Hou

Authors
  1. Yang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qianmu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shunmei Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qianmu Li .

Editor information

Editors and Affiliations

  1. Xiamen University, Xiamen, China

    Yongxuan Lai

  2. Beijing Normal University, Zhuhai, China

    Tian Wang

  3. Xiamen University, Xiamen, China

    Min Jiang

  4. Tianjin University, Tianjin, China

    Guangquan Xu

  5. Hunan University, Changsha, China

    Wei Liang

  6. University of Naples Parthenope, Naples, Italy

    Aniello Castiglione

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, Y., Li, Q., Meng, S., Hou, J. (2022). Transformer-Based Rating-Aware Sequential Recommendation. In: Lai, Y., Wang, T., Jiang, M., Xu, G., Liang, W., Castiglione, A. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2021. Lecture Notes in Computer Science(), vol 13155. Springer, Cham. https://doi.org/10.1007/978-3-030-95384-3_47

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-95384-3_47

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-95383-6

  • Online ISBN: 978-3-030-95384-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation