Abstract
Existing sequential recommendation methods focus on modeling the temporal relationships of users’ historical behaviors and excel in exploiting users’ dynamic interests to improve recommendation performance. However, these methods rarely consider the existence of multi-scale user behavior sequences (e.g., temporal, location, and material scales), and sometimes user multi-scale interests play a decisive role in predicting final user preferences. To investigate the influence of multi-scale interests on user preferences, we study to develop a Multi-scale Interest Dynamic Hierarchical Transformer Model (MIDHT) to fine-grain modeling of users’ interests. Specifically, the proposal includes: First, the neighbor attention mechanism determines whether two neighboring items merge or not. Second, we generate the block mask matrix based on the above judgment results. Third, we compute the implicit representation of the current layer using the dynamic block mask matrix and the self-attention mechanism. Last, the dynamic block mask matrix of all layers to infer the corresponding hierarchical structure. Thorough experiments are implemented to show the features of MIDHT under different component settings. Furthermore, experimental results on three real-world datasets show that MIDHT significantly outperforms the state-of-the-art baselines on different evaluation metrics.






Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
The time sequence used here is not a specific timestamp.
References
Meehan K, Lunney T, Curran K, McCaughey A (2013) Context-aware intelligent recommendation system for tourism. In: IEEE international conference on pervasive computing and communications workshops (PERCOM workshops), pp 328–331. IEEE, 2013
Chen Q, Zhao H, Li W, Huang P, Ou W (2019) Behavior sequence transformer for e-commerce recommendation in alibaba. In: Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data, pp 1–4
Rendle S, Freudenthaler C, Schmidt-Thieme L (2010) Factorizing personalized markov chains for next-basket recommendation. In: Proceedings of the 19th international conference on World wide web, pp 811–820
Liu Q, Zeng Y, Mokhosi R, Zhang H (2018) Stamp: short-term attention/memory priority model for session-based recommendation. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 1831–1839
Wang P, Guo J, Lan Y, Xu J, Wan S, Cheng X (2015) Learning hierarchical representation model for nextbasket recommendation. In: Proceedings of the 38th International ACM SIGIR conference on Research and Development in Information Retrieval, pp 403–412
Ren K, Qin J, Fang Y, Zhang W, Zheng L, Bian W, Zhou G, Xu J, Yu Y, Zhu X et al. (2019) Lifelong sequential modeling with personalized memorization for user response prediction. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 565–574
Yu L, Chuxu Z, Shangsong L, Xiangliang Z (2019) Multi-order attentive ranking model for sequential recommendation. In: Proceedings of the AAAI conference on Artificial Intelligence, vol 33, pp 5709–5716
Li S, Yang D, Zhang B (2020) MRIF: Multi-resolution interest fusion for recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 1765–1768
Hu Y, Koren Y, Volinsky C (2008) Collaborative filtering for implicit feedback datasets. In: 2008 Eighth IEEE International Conference on Data Mining, pp 263–272. IEEE
Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L (2012) Bpr: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618,
Linden G, Smith B, York J (2003) Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet comput, 7(1):76–80
Quadrana Massimo, Cremonesi Paolo, Jannach Dietmar (2018) Sequence-aware recommender systems. ACM Comput Surv (CSUR) 51(4):1–36
Yu F, Liu Q, Wu S, Wang L, Tan T (2016) A dynamic recurrent model for next basket recommendation. In: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pp 729–732
Kang W-C, McAuley J (2018) Self-attentive sequential recommendation. In: 2018 IEEE International Conference on Data Mining (ICDM), pp 197–206. IEEE
Harer J, Reale C, Chin P (2019) Tree-transformer: A transformer-based method for correction of tree-structured data. arXiv preprint arXiv:1908.00449
Hidasi B, Karatzoglou A, Baltrunas L, Tikk D (2015) Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939
Cho K, Van Merriënboer B, Bahdanau D, Bengio Y (2014) On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259
Zhao Wei, Wang Benyou, Yang Min, Ye Jianbo, Zhao Zhou, Chen Xiaojun, Shen Ying (2019) Leveraging long and short-term information in content-aware movie recommendation via adversarial training. IEEE Trans Cybern 50(11):4680–4693
Zhang S, Tay Y, Yao L, Sun A, An J (2019) Next item recommendation with self-attentive metric learning. In: Thirty-Third AAAI Conference on Artificial Intelligence, vol 9
Chen Z, Ma Q, Lin Z Time-aware multi-scale rnns for time series modeling
Koutnik J, Greff K, Gomez F, Schmidhuber J (2014) A clockwork RNN. In: International Conference on Machine Learning, pp 1863–1871. PMLR
Li J, Wang Y, McAuley J (2020) Time interval aware self-attention for sequential recommendation. In: Proceedings of the 13th international conference on web search and data mining, pp 322–330
Mingi J, Weonyoung J, Kyungwoo S, Yoon-Yeong K, Il-Chul M (2020) Sequential recommendation with relation-aware kernelized self-attention. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 4304–4311
Ren R, Liu Z, Li Y, Zhao X, Wang H, Ding B, Wen J-R (2020) Sequential recommendation with self-attentive multi-adversarial network. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, pp 89–98
Sarwar B, Karypis G, Konstan J, Riedl J (2001) Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th international conference on World Wide Web, pp 285–295
Zhang S, Tay Y, Yao L, Sun A (2018) Next item recommendation with self-attention. arXiv preprint arXiv:1808.06414
Deshpande Mukund, Karypis George (2004) Item-based top-n recommendation algorithms. ACM Trans Inf Syst (TOIS) 22(1):143–177
Salton G, Harman D (2003)Information retrieval. In: Encyclopedia of computer science, pp 858–863
Järvelin Kalervo, Kekäläinen Jaana (2002) Cumulated gain-based evaluation of IR techniques. ACM Trans Inf Syst (TOIS) 20(4):422–446
Liu L, Özsu MT (2009) Encyclopedia of database systems, vol 6. Springer, New York, NY, USA
Ma X, Zhao L, Huang G, Wang Z, Hu Z, Zhu X, Gai K (2018)Entire space multi-task model: an effective approach for estimating post-click conversion rate. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 1137–1140
Acknowledgements
This research was supported by the National Natural Science Foundation of China (No. U1736206). The numerical calculations in this paper have been done on the supercomputing system in the Supercomputing Center of Wuhan University.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no known competing financial interests or personal relationships that could have influenced the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Huang, N., Hu, R., Xiong, M. et al. Multi-scale Interest Dynamic Hierarchical Transformer for sequential recommendation. Neural Comput & Applic 34, 16643–16654 (2022). https://doi.org/10.1007/s00521-022-07281-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-022-07281-7