skip to main content
10.1145/3477495.3531846acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
short-paper

A New Sequential Prediction Framework with Spatial-temporal Embedding

Published: 07 July 2022 Publication History

Abstract

Sequential prediction is one of the key components in recommendation. In online e-commerce recommendation system, user behavior consists of the sequential visiting logs and item behavior contains the interacted user list in order. Most of the existing state-of-the-art sequential prediction methods only consider the user behavior while ignoring the item behavior. In addition, we find that user behavior varies greatly at different time, and most existing models fail to characterize the rich temporal information. To address the above problems, we propose a transformer-based spatial-temporal recommendation framework (STEM). In the STEM framework, we first utilize attention mechanisms to model user behavior and item behavior, and then exploit spatial and temporal information through a transformer-based model. The STEM framework, as a plug-in, is able to be incorporated into many neural network-based sequential recommendation methods to improve performance. We conduct extensive experiments on three real-world Amazon datasets. The results demonstrate the effectiveness of our proposed framework.

Supplementary Material

MP4 File (SIGIR22-sp1751.mp4)
Sequential prediction is one of the key components in recommendation. In online e-commerce recommendation system, user behavior consists of the sequential visiting logs and item behavior contains the interacted user list in order. Most of the existing state-of-the-art sequential prediction methods only consider the user behavior while ignoring the item behavior. In addition, we find that user behavior varies greatly at different time, and most existing models fail to characterize the rich temporal information. To address the above problems, we propose a transformer-based spatial-temporal recommendation framework (STEM). In the STEM framework, we first utilize attention mechanisms to model user behavior and item behavior, and then exploit spatial and temporal information through a transformer-based model. The STEM framework, as a plug-in, is able to be incorporated into many neural network-based sequential recommendation methods to improve performance.

References

[1]
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).
[2]
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems. 7--10.
[3]
Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems. 191--198.
[4]
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based recommendations with recurrent neural networks. In International Conference on Learning Representations.
[5]
Wei Jiang, Fangquan Lin, Jihai Zhang, Cheng Yang, Hanwei Zhang, and Ziqiang Cui. 2021. Dynamic Sequential Recommendation: Decoupling User Intent from Temporal Context. In ICDMW. IEEE, 18--26.
[6]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[7]
Xiang Li, Chao Wang, Bin Tong, Jiwei Tan, Xiaoyi Zeng, and Tao Zhuang. 2020. Deep Time-Aware Item Evolution Network for Click-Through Rate Prediction. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 785--794.
[8]
Yu Li, Liu Lu, and Li Xuefeng. 2005. A hybrid collaborative filtering method for multiple-interests and multiple-content recommendation in E-Commerce. Expert systems with applications 28, 1 (2005), 67--77.
[9]
Fangquan Lin, Wei Jiang, Jihai Zhang, and Cheng Yang. 2021. Dynamic popularityaware contrastive learning for recommendation. In ACML. PMLR, 964--968.
[10]
Chang Liu, Xiaoguang Li, Guohao Cai, Zhenhua Dong, Hong Zhu, and Lifeng Shang. 2021. Non-invasive Self-attention for Side Information Fusion in Sequential Recommendation. arXiv preprint arXiv:2103.03578 (2021).
[11]
Fuyu Lv, Taiwei Jin, Changlong Yu, Fei Sun, Quan Lin, Keping Yang, and Wilfred Ng. 2019. SDM: Sequential deep matching model for online large-scale recommender system. In Proceedings of the 28th ACM International Conference on Information & Knowledge Management. 2635--2643.
[12]
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015. Image-based recommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. 43--52.
[13]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111--3119.
[14]
Jianbo Ouyang, Hui Wu, Min Wang, Wengang Zhou, and Houqiang Li. 2021. Contextual Similarity Aggregation with Self-attention for Visual Re-ranking. Advances in Neural Information Processing Systems 34 (2021).
[15]
Qiaoyu Tan, Jianwei Zhang, Ninghao Liu, Xiao Huang, Hongxia Yang, Jingren Zhou, Xia Hu, et al. 2021. Dynamic memory based attention network for sequential recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4384--4392.
[16]
Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. 565--573.
[17]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. arXiv preprint arXiv:1706.03762 (2017).
[18]
S Wang, L Hu, Y Wang, L Cao, QZ Sheng, and M Orgun. 2019. Sequential recommender systems: Challenges, progress and prospects. In Twenty-Eighth International Joint Conference on Artificial Intelligence {IJCAI-19}. International Joint Conferences on Artificial Intelligence Organization.
[19]
Hui Wu, Min Wang, Wengang Zhou, Yang Hu, and Houqiang Li. 2021. Learning Token-based Representation for Image Retrieval. arXiv preprint arXiv:2112.06159 (2021).
[20]
Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys (CSUR) 52, 1 (2019), 1--38.
[21]
Chang Zhou, Jinze Bai, Junshuai Song, Xiaofei Liu, Zhengchao Zhao, Xiusi Chen, and Jun Gao. 2018. Atrank: An attention-based user behavior modeling framework for recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.
[22]
Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep interest evolution network for click-through rate prediction. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 5941--5948.
[23]
Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1059--1068.

Cited By

View all
  • (2024)Target - Attention Network for Click - Through Rate Prediction2024 IEEE 3rd International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA)10.1109/EEBDA60612.2024.10486031(366-370)Online publication date: 27-Feb-2024
  • (2022)An Enhanced Gated Graph Neural Network for E-commerce RecommendationProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557547(4677-4681)Online publication date: 17-Oct-2022

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval
July 2022
3569 pages
ISBN:9781450387323
DOI:10.1145/3477495
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. attention
  2. click-through rate
  3. sequential prediction
  4. spatial-temporal embedding

Qualifiers

  • Short-paper

Conference

SIGIR '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)37
  • Downloads (Last 6 weeks)5
Reflects downloads up to 28 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Target - Attention Network for Click - Through Rate Prediction2024 IEEE 3rd International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA)10.1109/EEBDA60612.2024.10486031(366-370)Online publication date: 27-Feb-2024
  • (2022)An Enhanced Gated Graph Neural Network for E-commerce RecommendationProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557547(4677-4681)Online publication date: 17-Oct-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media