Abstract
Sequential patterns and the order of items in sequences are particularly important for sequential recommendation (SR), which decide what item will interact with the user. However, there are still some problems for the existing methods: (1) They treat all the features of each item equally, we believe that those features the user pays more attention to of each item play a key role to predict next item. (2) Many methods not only ignore sequential patterns and the order of sequences, but also cannot highlight more important features, they only focus on whether the features exist. To address these issues, we propose the Sequential Patterns Enhanced Capsule Network (SPECN). SPECN leverages a self-attention mechanism, using user information as a guide to highlight the most relevant features for each item, then concatenates these features with the original item features in the sequence.SPECN applies horizontal and vertical capsule networks which package neurons into vectors to extract sequential patterns features and the order of sequences. The horizontal capsule network enhances sequential pattern features by learning both the original and user-focused features of individual or adjacent items, containing original features and those features that the user pays more attention to of single item or adjacent items’ features (features of the previous item that the users pay more attention to and features of the current item) to enhance the sequential patterns features. The vertical capsule network captures finer-grained feature representations for each item, improving the recommendation quality. We conduct several experiments on three real-world datasets to demonstrate the superiority of SPECN, outperforming existing methods in terms of accuracy and robustness.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
All relevant data are within the paper.
Code Availability
The data preprocessed and code of SPECN are available. Please click on this link: (https://github.com/ZhizhongZheng).
References
Fan Z, Liu Z, Zhang J, Xiong Y, Zheng L, Yu PS (2021) Continuous-time sequential recommendation with temporal graph collaborative transformer. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp 433–442
Du Y, Peng Z, Niu J, Yan J (2022) A unified hierarchical attention framework for sequential recommendation by fusing long and short-term preferences. Expert Syst Appl 201:117102
Kang WC, McAuley J (2018) Self-attentive sequential recommendation. In: 2018 IEEE International Conference on Data Mining (ICDM), pp 197–206. IEEE
Lv F, Jin T, Yu C, Sun F, Lin Q, Yang K, Ng W (2019) Sdm: Sequential deep matching model for online large-scale recommender system. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp 2635–2643
Fan Z, Liu Z, Wang Y, Wang A, Nazari Z, Zheng L, Peng H, Yu PS (2022) Sequential recommendation via stochastic self-attention. Proceedings of the ACM Web Conference 2022:2036–2047
Tang J, Wang K (2018) Personalized top-n sequential recommendation via convolutional sequence embedding. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pp 565–573
Xu C, Zhao P, Liu Y, Xu J, S. Sheng VSS, Cui Z, Zhou X, Xiong H (2019) Recurrent convolutional neural network for sequential recommendation. In: The World Wide Web Conference, pp 3398–3404
Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. Adv Neural Inf Process Syst 30
Rendle S, Freudenthaler C, Schmidt-Thieme L (2010) Factorizing personalized markov chains for next-basket recommendation. In: Proceedings of the 19th International Conference on World Wide Web, pp 811–820
He R, McAuley J (2016) Fusing similarity models with markov chains for sparse sequential recommendation. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp 191–200. IEEE
Xu C, Zhao P, Liu Y, Sheng VS, Xu J, Zhuang F, Fang J, Zhou X (2019) Graph contextualized self-attention network for session-based recommendation. IJCAI 19:3940–3946
Zhang S, Tay Y, Yao L, Sun A, An J (2019) Next item recommendation with self-attentive metric learning. In: 33rd AAAI Conference on Artificial Intelligence vol 9
Li J et al (2023) Text is all you need: Learning language representations for sequential recommendation. In: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
Yue Z et al () Linear recurrent units for sequential recommendation. In: Proceedings of the 17th ACM International Conference on Web Search and Data Mining
Mazzia V, Salvetti F, Chiaberge M (2021) Efficient-capsnet: Capsule network with self-attention routing. Sci Rep 11(1):1–13
Byerly A, Kalganova T, Dear I (2021) No routing needed between capsules. Neurocomput 463:545–553
Yang J, Zhao P, Rong Y, Yan C, Li C, Ma H, Huang J (2021) Hierarchical graph capsule network. Proceedings of the AAAI Conference on Artificial Intelligence 35:10603–10611
Han Y, Gu P, Gao W, Xu G, Wu J (2021) Aspect-level sentiment capsule network for micro-video click-through rate prediction. World Wide Web 24(4):1045–1064
Li D, Hu B, Chen Q, Wang X, Qi Q, Wang L, Liu H (2021) Attentive capsule network for click-through rate and conversion rate prediction in online advertising. Knowl-Based Syst 211:106522
Li C, Quan C, Peng L, Qi Y, Deng Y, Wu L (2019) A capsule network for recommendation and explaining what you like and dislike. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 275–284
Zhang Q, Wu B, Sun Z, Ye Y (2022) Gating augmented capsule network for sequential recommendation. Knowl-Based Syst 247:108817
Cheng Y et al (2024) Accurate multi-interest modeling for sequential recommendation with attention and distillation capsule network. Expert Syst Appl 243:122887
Kannikaklang N, Thamviset W, Wongthanavasu S (2024) Bilstcan: A novel srs-based bidirectional long short-term capsule attention network for dynamic user preference and next-item recommendation. IEEE Access
Ma C, Kang P, Liu X (2019) Hierarchical gating networks for sequential recommendation. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 825–833
Zhang S, Yao D, Zhao Z, Chua TS, Wu F (2021) Causerec: Counterfactual user sequence synthesis for sequential recommendation. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 367–377
Funding
This work was supported by the National Natural Science Foundation of China under Grant No.51975505 and HeBei Natural Science Foundation under Grant No.G2021203010 & No.F2021203038. Meanwhile, it was supported by Innovation Capability Improvement Plan Project of Hebei Province (22567637H).
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conceptualization and methodology. Material preparation, data collection and analysis were performed by Shunpan Liang, Zhizhong Zheng, Jixiang Yang, Guozheng Zhang and Qianjin Kong. The model is structured by Zhizhong Zheng and Qianjin Kong. The first draft of the manuscript was written by Zhizhong Zheng and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of Interest
There is no conflict of interests.
Ethics Approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Shunpan, L., Zhizhong, Z., Guozheng, Z. et al. SPECN:sequential patterns enhanced capsule network for sequential recommendation. Appl Intell 55, 204 (2025). https://doi.org/10.1007/s10489-024-06159-6
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-06159-6