Skip to main content

Advertisement

SPECN:sequential patterns enhanced capsule network for sequential recommendation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Sequential patterns and the order of items in sequences are particularly important for sequential recommendation (SR), which decide what item will interact with the user. However, there are still some problems for the existing methods: (1) They treat all the features of each item equally, we believe that those features the user pays more attention to of each item play a key role to predict next item. (2) Many methods not only ignore sequential patterns and the order of sequences, but also cannot highlight more important features, they only focus on whether the features exist. To address these issues, we propose the Sequential Patterns Enhanced Capsule Network (SPECN). SPECN leverages a self-attention mechanism, using user information as a guide to highlight the most relevant features for each item, then concatenates these features with the original item features in the sequence.SPECN applies horizontal and vertical capsule networks which package neurons into vectors to extract sequential patterns features and the order of sequences. The horizontal capsule network enhances sequential pattern features by learning both the original and user-focused features of individual or adjacent items, containing original features and those features that the user pays more attention to of single item or adjacent items’ features (features of the previous item that the users pay more attention to and features of the current item) to enhance the sequential patterns features. The vertical capsule network captures finer-grained feature representations for each item, improving the recommendation quality. We conduct several experiments on three real-world datasets to demonstrate the superiority of SPECN, outperforming existing methods in terms of accuracy and robustness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

All relevant data are within the paper.

Code Availability

The data preprocessed and code of SPECN are available. Please click on this link: (https://github.com/ZhizhongZheng).

Notes

  1. https://grouplens.org/datasets/movielens/1m/

  2. https://snap.stanford.edu/data/loc-gowalla.html

  3. http://jmcauley.ucsd.edu/data/amazon/

References

  1. Fan Z, Liu Z, Zhang J, Xiong Y, Zheng L, Yu PS (2021) Continuous-time sequential recommendation with temporal graph collaborative transformer. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp 433–442

  2. Du Y, Peng Z, Niu J, Yan J (2022) A unified hierarchical attention framework for sequential recommendation by fusing long and short-term preferences. Expert Syst Appl 201:117102

    Article  Google Scholar 

  3. Kang WC, McAuley J (2018) Self-attentive sequential recommendation. In: 2018 IEEE International Conference on Data Mining (ICDM), pp 197–206. IEEE

  4. Lv F, Jin T, Yu C, Sun F, Lin Q, Yang K, Ng W (2019) Sdm: Sequential deep matching model for online large-scale recommender system. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp 2635–2643

  5. Fan Z, Liu Z, Wang Y, Wang A, Nazari Z, Zheng L, Peng H, Yu PS (2022) Sequential recommendation via stochastic self-attention. Proceedings of the ACM Web Conference 2022:2036–2047

    MATH  Google Scholar 

  6. Tang J, Wang K (2018) Personalized top-n sequential recommendation via convolutional sequence embedding. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pp 565–573

  7. Xu C, Zhao P, Liu Y, Xu J, S. Sheng VSS, Cui Z, Zhou X, Xiong H (2019) Recurrent convolutional neural network for sequential recommendation. In: The World Wide Web Conference, pp 3398–3404

  8. Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. Adv Neural Inf Process Syst 30

  9. Rendle S, Freudenthaler C, Schmidt-Thieme L (2010) Factorizing personalized markov chains for next-basket recommendation. In: Proceedings of the 19th International Conference on World Wide Web, pp 811–820

  10. He R, McAuley J (2016) Fusing similarity models with markov chains for sparse sequential recommendation. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp 191–200. IEEE

  11. Xu C, Zhao P, Liu Y, Sheng VS, Xu J, Zhuang F, Fang J, Zhou X (2019) Graph contextualized self-attention network for session-based recommendation. IJCAI 19:3940–3946

    MATH  Google Scholar 

  12. Zhang S, Tay Y, Yao L, Sun A, An J (2019) Next item recommendation with self-attentive metric learning. In: 33rd AAAI Conference on Artificial Intelligence vol 9

  13. Li J et al (2023) Text is all you need: Learning language representations for sequential recommendation. In: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

  14. Yue Z et al () Linear recurrent units for sequential recommendation. In: Proceedings of the 17th ACM International Conference on Web Search and Data Mining

  15. Mazzia V, Salvetti F, Chiaberge M (2021) Efficient-capsnet: Capsule network with self-attention routing. Sci Rep 11(1):1–13

    Article  Google Scholar 

  16. Byerly A, Kalganova T, Dear I (2021) No routing needed between capsules. Neurocomput 463:545–553

    Article  Google Scholar 

  17. Yang J, Zhao P, Rong Y, Yan C, Li C, Ma H, Huang J (2021) Hierarchical graph capsule network. Proceedings of the AAAI Conference on Artificial Intelligence 35:10603–10611

    Article  Google Scholar 

  18. Han Y, Gu P, Gao W, Xu G, Wu J (2021) Aspect-level sentiment capsule network for micro-video click-through rate prediction. World Wide Web 24(4):1045–1064

    Article  Google Scholar 

  19. Li D, Hu B, Chen Q, Wang X, Qi Q, Wang L, Liu H (2021) Attentive capsule network for click-through rate and conversion rate prediction in online advertising. Knowl-Based Syst 211:106522

    Article  MATH  Google Scholar 

  20. Li C, Quan C, Peng L, Qi Y, Deng Y, Wu L (2019) A capsule network for recommendation and explaining what you like and dislike. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 275–284

  21. Zhang Q, Wu B, Sun Z, Ye Y (2022) Gating augmented capsule network for sequential recommendation. Knowl-Based Syst 247:108817

    Article  MATH  Google Scholar 

  22. Cheng Y et al (2024) Accurate multi-interest modeling for sequential recommendation with attention and distillation capsule network. Expert Syst Appl 243:122887

    Article  Google Scholar 

  23. Kannikaklang N, Thamviset W, Wongthanavasu S (2024) Bilstcan: A novel srs-based bidirectional long short-term capsule attention network for dynamic user preference and next-item recommendation. IEEE Access

  24. Ma C, Kang P, Liu X (2019) Hierarchical gating networks for sequential recommendation. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 825–833

  25. Zhang S, Yao D, Zhao Z, Chua TS, Wu F (2021) Causerec: Counterfactual user sequence synthesis for sequential recommendation. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 367–377

Download references

Funding

This work was supported by the National Natural Science Foundation of China under Grant No.51975505 and HeBei Natural Science Foundation under Grant No.G2021203010 & No.F2021203038. Meanwhile, it was supported by Innovation Capability Improvement Plan Project of Hebei Province (22567637H).

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the study conceptualization and methodology. Material preparation, data collection and analysis were performed by Shunpan Liang, Zhizhong Zheng, Jixiang Yang, Guozheng Zhang and Qianjin Kong. The model is structured by Zhizhong Zheng and Qianjin Kong. The first draft of the manuscript was written by Zhizhong Zheng and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Liang Shunpan.

Ethics declarations

Conflict of Interest

There is no conflict of interests.

Ethics Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shunpan, L., Zhizhong, Z., Guozheng, Z. et al. SPECN:sequential patterns enhanced capsule network for sequential recommendation. Appl Intell 55, 204 (2025). https://doi.org/10.1007/s10489-024-06159-6

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-06159-6

Keywords