skip to main content
research-article

Multi-Hop Multi-View Memory Transformer for Session-Based Recommendation

Published: 11 July 2024 Publication History

Abstract

A Session-Based Recommendation (SBR) seeks to predict users’ future item preferences by analyzing their interactions with previously clicked items. In recent approaches, Graph Neural Networks (GNNs) have been commonly applied to capture item relations within a session to infer user intentions. However, these GNN-based methods typically struggle with feature ambiguity between the sequential session information and the item conversion within an item graph, which may impede the model’s ability to accurately infer user intentions. In this article, we propose a novel Multi-hop Multi-view Memory Transformer (M3T) to effectively integrate the sequence-view information and relation conversion (graph-view information) of items in a session. First, we propose a Multi-view Memory Transformer (M2T) module to concurrently obtain multi-view information of items. Then, a set of trainable memory matrices are employed to store sharable item features, which mitigates cross-view item feature ambiguity. To comprehensively capture latent user intentions, an M3T framework is designed to integrate user intentions across different hops of an item graph. Specifically, a k-order power method is proposed to manage the item graph to alleviate the over-smoothing problem when obtaining high-order relations of items. Extensive experiments conducted on three real-world datasets demonstrate the superiority of our method.

References

[1]
Gediminas Adomavicius and Alexander Tuzhilin. 2005. Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions. IEEE TKDE 17, 6 (2005), 734–749.
[2]
Fidel Cacheda, Victor Carneiro, Diego Fernández, and Vreixo Formoso. 2011. Comparison of Collaborative Filtering Algorithms: Limitations of Current Techniques and Proposals for Scalable, High-performance Recommender systems. ACM TWEB 5, 1 (2011), 2:1–2:33.
[3]
Chenwei Cai, Ruining He, and Julian J. McAuley. 2017. SPMC: Socially-Aware Personalized Markov Chains for Sparse Sequential Recommendation. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI ’17). ijcai.org, 1476–1482.
[4]
Desheng Cai, Shengsheng Qian, Quan Fang, Jun Hu, and Changsheng Xu. 2023. User Cold-Start Recommendation via Inductive Heterogeneous Graph Neural Network. ACM TOIS 41, 3 (2023), 64 1–64:27.
[5]
Tianwen Chen and Raymond Chi-Wing Wong. 2020. Handling Information Loss of Graph Neural Networks for Session-Based Recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’20). ACM, 1172–1180.
[6]
Kyunghyun Cho, Bart van Merrienboer, Caglar Gülcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP ’14). ACL, 1724–1734.
[7]
Ranak Roy Chowdhury, Xiyuan Zhang, Jingbo Shang, Rajesh K. Gupta, and Dezhi Hong. 2022. TARNet: Task-Aware Reconstruction for Time-Series Transformer. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’22). ACM, 212–220.
[8]
Verna Dankers, Christopher G. Lucas, and Ivan Titov. 2022. Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL ’22), Vol. 1. Association for Computational Linguistics, 3608–3626.
[9]
Alexandre de Brébisson, Étienne Simon, Alex Auvolat, Pascal Vincent, and Yoshua Bengio. 2015. Artificial Neural Networks Applied to Taxi Destination Prediction. In Proceedings of the International Conference on ECML/PKDD Discovery Challenge (ECMLPKDDDC ’15), Vol. 1526. CEUR-WS.org, 1–12.
[10]
Ricardo Dias and Manuel J. Fonseca. 2013. Improving Music Recommendation in Session-Based Collaborative Filtering by Using Temporal Context. In Proceedings of the 25th IEEE International Conference on Tools with Artificial Intelligence (ICTAI ’13). IEEE Computer Society, 783–788.
[11]
William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. JMLR 23 (2022), 120 1–120:39.
[12]
Jiayan Guo, Yaming Yang, Xiangchen Song, Yuan Zhang, Yujing Wang, Jing Bai, and Yan Zhang. 2022. Learning Multi-granularity Consecutive User Intent Unit for Session-Based Recommendation. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining (WSDM ’22). ACM, 343–352.
[13]
Lei Guo, Hongzhi Yin, Qinyong Wang, Tong Chen, Alexander Zhou, and Nguyen Quoc Viet Hung. 2019. Streaming Session-Based Recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’19). ACM, 1569–1577.
[14]
Qingyu Guo, Fuzhen Zhuang, Chuan Qin, Hengshu Zhu, Xing Xie, Hui Xiong, and Qing He. 2022. A Survey on Knowledge Graph-Based Recommender Systems. IEEE TKDE 34, 8 (2022), 3549–3568.
[15]
Ruining He, Chen Fang, Zhaowen Wang, and Julian J. McAuley. 2016. Vista: A Visually, Socially, and Temporally-aware Model for Artistic Recommendation. In Proceedings of the 10th ACM Conference on Recommender Systems (RecSys ’16). ACM, 309–316.
[16]
Ruining He and Julian J. McAuley. 2016. Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation. In Proceedings of the 16th IEEE International Conference on Data Mining (ICDM ’16). IEEE Computer Society, 191–200.
[17]
Balázs Hidasi and Alexandros Karatzoglou. 2018. Recurrent Neural Networks with Top-k Gains for Session-Based Recommendations. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (CIKM ’18). ACM, 843–852.
[18]
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-Based Recommendations with Recurrent Neural Networks. In Proceedings of the 4th International Conference on Learning Representations (ICLR ’16). OpenReview.net.
[19]
Cheng Hsu and Cheng-Te Li. 2021. RetaGNN: Relational Temporal Attentive Graph Neural Networks for Holistic Sequential Recommendation. In Proceedings of the ACM of Web Conference (WWW ’21). ACM/IW3C2, 2968–2979.
[20]
Jun Hu, Shengsheng Qian, Quan Fang, Youze Wang, Quan Zhao, Huaiwen Zhang, and Changsheng Xu. 2021. Efficient Graph Deep Learning in TensorFlow with tf_geometric. In Proceedings of the 29th ACM International Conference on Multimedia (MM ’21). ACM, 3775–3778.
[21]
Wang-Cheng Kang and Julian J. McAuley. 2018. Self-Attentive Sequential Recommendation. In Proceedings of IEEE International Conference on Data Mining (ICDM ’18). IEEE Computer Society, 197–206.
[22]
Yang Sok Kim, Hyunwoo Hwangbo, Hee Jun Lee, and Won Seok Lee. 2022. Sequence Aware Recommenders for Fashion E-commerce (Published Online). Electron. Commer. Res. (2022), 1–21.
[23]
Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In Proceedings of the 5th International Conference on Learning Representations (ICLR ’17). OpenReview.net.
[24]
Siqi Lai, Erli Meng, Fan Zhang, Chenliang Li, Bin Wang, and Aixin Sun. 2022. An Attribute-Driven Mirror Graph Network for Session-Based Recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’22). ACM, 1674–1683.
[25]
David Lenz, Christian Schulze, and Michael Guckert. 2018. Real-Time Session-Based Recommendations Using LSTM with Neural Embeddings. In Proceedings of the 27th International Conference on Artificial Neural Networks (ICANN ’18), Vol. 11140. Springer, 337–348.
[26]
Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural Attentive Session-Based Recommendation. In Proceedings of the 26th ACM on Conference on Information and Knowledge Management (CIKM ’17). ACM, 1419–1428.
[27]
Jiacheng Li, Yujie Wang, and Julian J. McAuley. 2020. Time Interval Aware Self-Attention for Sequential Recommendation. In Proceedings of the 13th ACM International Conference on Web Search and Data Mining (WSDM ’20). ACM, 322–330.
[28]
Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard S. Zemel. 2016. Gated Graph Sequence Neural Networks. In Proceedings of the 4th International Conference on Learning Representations (ICLR ’16). OpenReview.net.
[29]
Qiao Liu, Yifu Zeng, Refuoe Mokhosi, and Haibin Zhang. 2018. STAMP: Short-Term Attention/Memory Priority Model for Session-Based Recommendation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’18). ACM, 1831–1839.
[30]
Yong Liu, Susen Yang, Yonghui Xu, Chunyan Miao, Min Wu, and Juyong Zhang. 2023. Contextualized Graph Attention Network for Recommendation With Item Knowledge Graph. IEEE TKDE 35, 1 (2023), 181–195.
[31]
Yu Lu, Jiali Zeng, Jiajun Zhang, Shuangzhi Wu, and Mu Li. 2022. Learning Confidence for Transformer-based Neural Machine Translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL ’22). Association for Computational Linguistics, 2353–2364.
[32]
Chen Ma, Liheng Ma, Yingxue Zhang, Jianing Sun, Xue Liu, and Mark Coates. 2021. Memory Augmented Graph Neural Networks for Sequential Recommendation. In Proceedings of the 34th Conference on Artificial Intelligence (AAAI ’21). AAAI Press, 5045–5052.
[33]
Loubna Mekouar, Youssef Iraqi, Issam W. Damaj, and Tarek Naous. 2022. A Survey on Blockchain-Based Recommender Systems: Integration Architecture and Taxonomy. Comput. Commun. 187 (2022), 1–19.
[34]
Wenjing Meng, Deqing Yang, and Yanghua Xiao. 2020. Incorporating User Micro-Behaviors and Item Knowledge into Multi-Task Learning for Session-Based Recommendation. In Proceedings of the 43rd ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’20). ACM, 1091–1100.
[35]
Tomás Mikolov, Martin Karafiát, Lukás Burget, Jan Cernocký, and Sanjeev Khudanpur. 2010. Recurrent Neural Network Based Language Model. In Proceedings of the 11th Annual International Symposium on Computer Architecture (ISCA ’10). ISCA, 1045–1048.
[36]
Zhiqiang Pan, Fei Cai, Wanyu Chen, Chonghao Chen, and Honghui Chen. 2022. Collaborative Graph Learning for Session-Based Recommendation. ACM TOIS 40, 4 (2022), 72:1–72:26.
[37]
Zhiqiang Pan, Fei Cai, Wanyu Chen, and Honghui Chen. 2022. Graph Co-Attentive Session-Based Recommendation. ACM TOIS 40, 4 (2022), 67:1–67:31.
[38]
Zhiqiang Pan, Fei Cai, Wanyu Chen, Honghui Chen, and Maarten de Rijke. 2020. Star Graph Neural Networks for Session-Based Recommendation. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (CIKM ’20). ACM, 1195–1204.
[39]
Ruihong Qiu, Zi Huang, Tong Chen, and Hongzhi Yin. 2022. Exploiting Positional Information for Session-Based Recommendation. ACM TOIS 40, 2 (2022), 35:1–35:24.
[40]
Ruihong Qiu, Zi Huang, Jingjing Li, and Hongzhi Yin. 2020. Exploiting Cross-Session Information for Session-Based Recommendation with Graph Neural Networks. ACM TOIS 38, 3 (2020), 22:1–22:23.
[41]
Ruihong Qiu, Jingjing Li, Zi Huang, and Hongzhi Yin. 2019. Rethinking the Item Order in Session-Based Recommendation with Graph Neural Networks. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM ’19). ACM, 579–588.
[42]
Pengjie Ren, Zhumin Chen, Jing Li, Zhaochun Ren, Jun Ma, and Maarten de Rijke. 2019. RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-Based Recommendation. In Proceedings of the 33rd Conference on Artificial Intelligence (AAAI ’19). AAAI Press, 4806–4813.
[43]
Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing Personalized Markov Chains for Next-basket Recommendation. In Proceedings of the 19th ACM International Conference on World Wide Web (WWW ’10). ACM, 811–820.
[44]
Badrul Munir Sarwar, George Karypis, Joseph A. Konstan, and John Riedl. 2001. Item-Based Collaborative Filtering Recommendation Algorithms. In Proceedings of the 10th ACM International Conference on World Wide Web (WWW ’01). ACM, 285–295.
[45]
Imanol Schlag, Kazuki Irie, and Jürgen Schmidhuber. 2021. Linear Transformers Are Secretly Fast Weight Programmers. In Proceedings of the 38th International Conference on Machine Learning (ICML ’21), Vol. 139. PMLR, 9355–9366.
[46]
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM ’19). ACM, 1441–1450.
[47]
Yong Kiam Tan, Xinxing Xu, and Yong Liu. 2016. Improved Recurrent Neural Networks for Session-Based Recommendations. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems (DLRS@RecSys ’16). ACM, 17–22.
[48]
Trinh X. Tuan and Tu M. Phuong. 2017. 3D Convolutional Networks for Session-Based Recommendation with Content Features. In Proceedings of the 11th ACM Conference on Recommender Systems (RecSys ’17). ACM, 138–146.
[49]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. In Proceedings of the 31st Advances in Neural Information Processing Systems (NIPS ’17). Curran Associates, Inc., 5998–6008.
[50]
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph Attention Networks. In Proceedings of the 6th International Conference on Learning Representations (ICLR ’18). OpenReview.net.
[51]
Minh N. Vu and My T. Thai. 2020. PGM-Explainer: Probabilistic Graphical Model Explanations for Graph Neural Networks. In Proceedings of the 34th International Conference on Neural Information Processing Systems (NIPS ’20). Curran Associates, Inc., 12225–12235.
[52]
Jianling Wang, Kaize Ding, Ziwei Zhu, and James Caverlee. 2021. Session-based Recommendation with Hypergraph Attention Networks. In Proceedings of the SIAM International Conference on Data Mining (SDM ’21). SIAM, 82–90.
[53]
Jinshan Wang, Qianfang Xu, Jiahuan Lei, Chaoqun Lin, and Bo Xiao. 2020. PA-GGAN: Session-Based Recommendation with Position-Aware Gated Graph Attention Network. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME ’20). IEEE, 1–6.
[54]
Shoujin Wang, Longbing Cao, Yan Wang, Quan Z. Sheng, Mehmet A. Orgun, and Defu Lian. 2022. A Survey on Session-Based Recommender Systems. ACM Comput. Surv. 54, 7 (2022), 154 1–154:38.
[55]
Wenjie Wang, Xinyu Lin, Liuhui Wang, Fuli Feng, Yinwei Wei, and Tat-Seng Chua. 2023. Equivariant Learning for Out-of-Distribution Cold-start Recommendation. In Proceedings of the 31st ACM International Conference on Multimedia (MM ’23). ACM, 903–914.
[56]
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. 2019. Heterogeneous Graph Attention Network. In Proceedings of the ACM World Wide Web Conference (WWW ’19). ACM, 2022–2032.
[57]
Yikai Wang, Xinghao Chen, Lele Cao, Wenbing Huang, Fuchun Sun, and Yunhe Wang. 2022. Multimodal Token Fusion for Vision Transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR ’22). IEEE, 12176–12185.
[58]
Ziyang Wang, Wei Wei, Gao Cong, Xiao-Li Li, Xianling Mao, and Minghui Qiu. 2020. Global Context Enhanced Graph Neural Networks for Session-Based Recommendation. In Proceedings of the 43rd ACM SIGIR Conference on Research and Development in Information (SIGIR ’20). ACM, 169–178.
[59]
Chunyu Wei, Bing Bai, Kun Bai, and Fei Wang. 2022. GSL4Rec: Session-Based Recommendations with Collective Graph Structure Learning and Next Interaction Prediction. In Proceedings of ACM Web Conference (WWW ’22). ACM, 2120–2130.
[60]
Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019. Session-Based Recommendation with Graph Neural Networks. In Proceedings of the 33rd Conference on Artificial Intelligence (AAAI ’19). AAAI Press, 346–353.
[61]
Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Lizhen Cui, and Xiangliang Zhang. 2021. Self-Supervised Hypergraph Convolutional Networks for Session-Based Recommendation. In Proceedings of the 35th Conference on Artificial Intelligence (AAAI ’21). AAAI Press, 4503–4511.
[62]
Chengfeng Xu, Pengpeng Zhao, Yanchi Liu, Victor S. Sheng, Jiajie Xu, Fuzhen Zhuang, Junhua Fang, and Xiaofang Zhou. 2019. Graph Contextualized Self-Attention Network for Session-Based Recommendation. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI ’19). ijcai.org, 3940–3946.
[63]
Dizhan Xue, Shengsheng Qian, Quan Fang, and Changsheng Xu. 2022. MMT: Image-Guided Story Ending Generation with Multimodal Memory Transformer. In Proceedings of the 30th ACM International Conference on Multimedia (MM ’22). ACM, 750–758.
[64]
Feng Yu, Yanqiao Zhu, Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. 2020. TAGNN: Target Attentive Graph Neural Networks for Session-Based Recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’20). ACM, 1921–1924.
[65]
George Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. 2021. A Transformer-Based Framework for Multivariate Time Series Representation Learning. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD ’21). ACM, 2114–2124.
[66]
Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020. S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (CIKM ’20). ACM, 1893–1902.
[67]
Chenxu Zhu, Peng Du, Xianghui Zhu, Weinan Zhang, Yong Yu, and Yang Cao. 2022. User-Tag Profile Modeling in Recommendation System via Contrast Weighted Tag Masking. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’22). ACM, 4630–4638.

Cited By

View all
  • (2025)Dual channel representation-learning with dynamic intent aggregation for session-based recommendationExpert Systems with Applications10.1016/j.eswa.2024.125273259(125273)Online publication date: Jan-2025

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Information Systems
ACM Transactions on Information Systems  Volume 42, Issue 6
November 2024
813 pages
EISSN:1558-2868
DOI:10.1145/3618085
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2024
Online AM: 08 May 2024
Accepted: 22 April 2024
Revised: 21 February 2024
Received: 12 November 2022
Published in TOIS Volume 42, Issue 6

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Session-based recommendation
  2. multi-view intention fusion
  3. memory Transformer
  4. multi-hop graph embedding

Qualifiers

  • Research-article

Funding Sources

  • Beijing Natural Science Foundation
  • University of the Ministry of Education
  • National Natural Science Foundation of China

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)463
  • Downloads (Last 6 weeks)44
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Dual channel representation-learning with dynamic intent aggregation for session-based recommendationExpert Systems with Applications10.1016/j.eswa.2024.125273259(125273)Online publication date: Jan-2025

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media