Skip to main content

Advertisement

Log in

Multi-scale Interest Dynamic Hierarchical Transformer for sequential recommendation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Existing sequential recommendation methods focus on modeling the temporal relationships of users’ historical behaviors and excel in exploiting users’ dynamic interests to improve recommendation performance. However, these methods rarely consider the existence of multi-scale user behavior sequences (e.g., temporal, location, and material scales), and sometimes user multi-scale interests play a decisive role in predicting final user preferences. To investigate the influence of multi-scale interests on user preferences, we study to develop a Multi-scale Interest Dynamic Hierarchical Transformer Model (MIDHT) to fine-grain modeling of users’ interests. Specifically, the proposal includes: First, the neighbor attention mechanism determines whether two neighboring items merge or not. Second, we generate the block mask matrix based on the above judgment results. Third, we compute the implicit representation of the current layer using the dynamic block mask matrix and the self-attention mechanism. Last, the dynamic block mask matrix of all layers to infer the corresponding hierarchical structure. Thorough experiments are implemented to show the features of MIDHT under different component settings. Furthermore, experimental results on three real-world datasets show that MIDHT significantly outperforms the state-of-the-art baselines on different evaluation metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. The time sequence used here is not a specific timestamp.

  2. http://jmcauley.ucsd.edu/data/amazon/.

  3. https://grouplens.org/datasets/movielens/1m/.

  4. https://webscope.sandbox.yahoo.com/catalog.php?datatype=r.

References

  1. Meehan K, Lunney T, Curran K, McCaughey A (2013) Context-aware intelligent recommendation system for tourism. In: IEEE international conference on pervasive computing and communications workshops (PERCOM workshops), pp 328–331. IEEE, 2013

  2. Chen Q, Zhao H, Li W, Huang P, Ou W (2019) Behavior sequence transformer for e-commerce recommendation in alibaba. In: Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data, pp 1–4

  3. Rendle S, Freudenthaler C, Schmidt-Thieme L (2010) Factorizing personalized markov chains for next-basket recommendation. In: Proceedings of the 19th international conference on World wide web, pp 811–820

  4. Liu Q, Zeng Y, Mokhosi R, Zhang H (2018) Stamp: short-term attention/memory priority model for session-based recommendation. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 1831–1839

  5. Wang P, Guo J, Lan Y, Xu J, Wan S, Cheng X (2015) Learning hierarchical representation model for nextbasket recommendation. In: Proceedings of the 38th International ACM SIGIR conference on Research and Development in Information Retrieval, pp 403–412

  6. Ren K, Qin J, Fang Y, Zhang W, Zheng L, Bian W, Zhou G, Xu J, Yu Y, Zhu X et al. (2019) Lifelong sequential modeling with personalized memorization for user response prediction. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 565–574

  7. Yu L, Chuxu Z, Shangsong L, Xiangliang Z (2019) Multi-order attentive ranking model for sequential recommendation. In: Proceedings of the AAAI conference on Artificial Intelligence, vol 33, pp 5709–5716

  8. Li S, Yang D, Zhang B (2020) MRIF: Multi-resolution interest fusion for recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 1765–1768

  9. Hu Y, Koren Y, Volinsky C (2008) Collaborative filtering for implicit feedback datasets. In: 2008 Eighth IEEE International Conference on Data Mining, pp 263–272. IEEE

  10. Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L (2012) Bpr: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618,

  11. Linden G, Smith B, York J (2003) Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet comput, 7(1):76–80

  12. Quadrana Massimo, Cremonesi Paolo, Jannach Dietmar (2018) Sequence-aware recommender systems. ACM Comput Surv (CSUR) 51(4):1–36

    Article  Google Scholar 

  13. Yu F, Liu Q, Wu S, Wang L, Tan T (2016) A dynamic recurrent model for next basket recommendation. In: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pp 729–732

  14. Kang W-C, McAuley J (2018) Self-attentive sequential recommendation. In: 2018 IEEE International Conference on Data Mining (ICDM), pp 197–206. IEEE

  15. Harer J, Reale C, Chin P (2019) Tree-transformer: A transformer-based method for correction of tree-structured data. arXiv preprint arXiv:1908.00449

  16. Hidasi B, Karatzoglou A, Baltrunas L, Tikk D (2015) Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939

  17. Cho K, Van Merriënboer B, Bahdanau D, Bengio Y (2014) On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259

  18. Zhao Wei, Wang Benyou, Yang Min, Ye Jianbo, Zhao Zhou, Chen Xiaojun, Shen Ying (2019) Leveraging long and short-term information in content-aware movie recommendation via adversarial training. IEEE Trans Cybern 50(11):4680–4693

    Article  Google Scholar 

  19. Zhang S, Tay Y, Yao L, Sun A, An J (2019) Next item recommendation with self-attentive metric learning. In: Thirty-Third AAAI Conference on Artificial Intelligence, vol 9

  20. Chen Z, Ma Q, Lin Z Time-aware multi-scale rnns for time series modeling

  21. Koutnik J, Greff K, Gomez F, Schmidhuber J (2014) A clockwork RNN. In: International Conference on Machine Learning, pp 1863–1871. PMLR

  22. Li J, Wang Y, McAuley J (2020) Time interval aware self-attention for sequential recommendation. In: Proceedings of the 13th international conference on web search and data mining, pp 322–330

  23. Mingi J, Weonyoung J, Kyungwoo S, Yoon-Yeong K, Il-Chul M (2020) Sequential recommendation with relation-aware kernelized self-attention. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 4304–4311

  24. Ren R, Liu Z, Li Y, Zhao X, Wang H, Ding B, Wen J-R (2020) Sequential recommendation with self-attentive multi-adversarial network. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, pp 89–98

  25. Sarwar B, Karypis G, Konstan J, Riedl J (2001) Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th international conference on World Wide Web, pp 285–295

  26. Zhang S, Tay Y, Yao L, Sun A (2018) Next item recommendation with self-attention. arXiv preprint arXiv:1808.06414

  27. Deshpande Mukund, Karypis George (2004) Item-based top-n recommendation algorithms. ACM Trans Inf Syst (TOIS) 22(1):143–177

    Article  Google Scholar 

  28. Salton G, Harman D (2003)Information retrieval. In: Encyclopedia of computer science, pp 858–863

  29. Järvelin Kalervo, Kekäläinen Jaana (2002) Cumulated gain-based evaluation of IR techniques. ACM Trans Inf Syst (TOIS) 20(4):422–446

    Article  Google Scholar 

  30. Liu L, Özsu MT (2009) Encyclopedia of database systems, vol 6. Springer, New York, NY, USA

  31. Ma X, Zhao L, Huang G, Wang Z, Hu Z, Zhu X, Gai K (2018)Entire space multi-task model: an effective approach for estimating post-click conversion rate. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 1137–1140

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (No. U1736206). The numerical calculations in this paper have been done on the supercomputing system in the Supercomputing Center of Wuhan University.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ruimin Hu.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have influenced the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, N., Hu, R., Xiong, M. et al. Multi-scale Interest Dynamic Hierarchical Transformer for sequential recommendation. Neural Comput & Applic 34, 16643–16654 (2022). https://doi.org/10.1007/s00521-022-07281-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07281-7

Keywords