skip to main content
research-article

Modeling Long- and Short-Term User Preferences via Self-Supervised Learning for Next POI Recommendation

Published: 15 June 2023 Publication History

Abstract

With the accumulation of check-in data from location-based services, next Point-of-Interest (POI) recommendations are gaining increasing attention. It is well known that the spatio-temporal contextual information of user check-in behavior plays a crucial role in handling vital and inherent challenges in next POI recommendation, including capture of user dynamic preferences and the sparsity problem of check-in data. However, many studies either ignore or simply stack the context features with the embedding of POIs while relying only on POI recommendation loss to optimize the entire model, therefore failing to take full advantage of the potential information in contexts. Additionally, users’ interests are usually unstable and evolve over time, and accordingly recent studies have proposed various approaches to predict users’ next POIs by incorporating contextual information and modeling both their long- and short-term preferences, respectively. Yet many studies overemphasize the final POI recommendation performance, and the association between POI sequences and contextual information is not well embodied in data representations. In this article, we focus on the preceding problems and propose a unified attention framework for next POI recommendation by modeling users’ Long- and Short-term Preferences via Self-supervised Learning (LSPSL). Specifically, based on the self-attention network and two self-supervised optimization objectives, LSPSL first deeply exploits the intrinsic correlations between POI sequences and contextual information through pre-training, which strengthens data representations. Then, supported by pre-trained contextualized embeddings, LSPSL models and fuses users’ complex long- and short-term preferences in a unified way. Extensive experiments on real-world datasets demonstrate the superiority of our model compared with other state-of-the-art approaches.

References

[1]
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450 (2016).
[2]
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Proceedings of the International Conference on Machine Learning. 1597–1607.
[3]
Chen Cheng, Haiqin Yang, Michael R. Lyu, and Irwin King. 2013. Where you like to go next: Successive point-of-interest recommendation. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence. 2605–2611.
[4]
Shaojie Dai, Yanwei Yu, Hao Fan, and Junyu Dong. 2022. Spatio-temporal representation learning with social tie for personalized POI recommendation. Data Science and Engineering 7, 1 (2022), 44–56.
[5]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4171–4186.
[6]
Jinfeng Fang, Xiangfu Meng, and Xueyue Qi. 2023. A top-k POI recommendation approach based on LBSN and multi-graph fusion. Neurocomputing 518 (2023), 219–230.
[7]
Jie Feng, Yong Li, Chao Zhang, Funing Sun, Fanchao Meng, Ang Guo, and Depeng Jin. 2018. DeepMove: Predicting human mobility with attentional recurrent networks. In Proceedings of the 2018 World Wide Web Conference. 1459–1468.
[8]
Shanshan Feng, Xutao Li, Yifeng Zeng, Gao Cong, Yeow Meng Chee, and Quan Yuan. 2015. Personalized ranking metric embedding for next new POI recommendation. In Proceedings of the 24th International Conference on Artificial Intelligence. 2069–2075.
[9]
Mingxin Gan and Yingxue Ma. 2023. Mapping user interest into hyper-spherical space: A novel POI recommendation method. Information Processing & Management 60, 2 (2023), 103169.
[10]
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9729–9738.
[11]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.
[12]
R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. 2019. Learning deep representations by mutual information estimation and maximization. In Proceedings of the International Conference on Learning Representations.
[13]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735–1780.
[14]
Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya Banerjee, and Fillia Makedon. 2020. A survey on contrastive self-supervised learning. Technologies 9, 1 (2020), 2.
[15]
Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In Proceedings of the 2018 IEEE International Conference on Data Mining (ICDM’18). 197–206.
[16]
Dejiang Kong and Fei Wu. 2018. HST-LSTM: A hierarchical spatial-temporal long-short term memory network for location prediction. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2341–2347.
[17]
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. ALBERT: A lite BERT for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019).
[18]
Jiacheng Li, Yujie Wang, and Julian McAuley. 2020. Time interval aware self-attention for sequential recommendation. In Proceedings of the 13th International Conference on Web Search and Data Mining. ACM, New York, NY, 322–330.
[19]
Yang Li, Tong Chen, Peng-Fei Zhang, Zi Huang, and Hongzhi Yin. 2022. Self-supervised graph-based point-of-interest recommendation. arXiv preprint arXiv:2210.12506 (2022).
[20]
Defu Lian, Yongji Wu, Yong Ge, Xing Xie, and Enhong Chen. 2020. Geography-aware sequential location recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, 2009–2019.
[21]
Dongliang Liao, Weiqing Liu, Yuan Zhong, Jing Li, and Guowei Wang. 2018. Predicting activity and location with multi-task context aware recurrent neural network. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. 3435–3441.
[22]
Yan Lin, Huaiyu Wan, Shengnan Guo, and Youfang Lin. 2021. Pre-training context and time aware location embeddings from spatial-temporal trajectories for user next location prediction. In Proceedings of the AAAI Conference on Artificial Intelligence. 4241–4248.
[23]
Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. 2016. Predicting the next location: A recurrent model with spatial and temporal contexts. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. 194–200.
[24]
Xiao Liu, Fanjin Zhang, Zhenyu Hou, Li Mian, Zhaoyu Wang, Jing Zhang, and Jie Tang. 2023. Self-supervised learning: Generative or contrastive. IEEE Transactions on Knowledge and Data Engineering 35, 1 (2023), 857–876.
[25]
Zhiwei Liu, Yongjun Chen, Jia Li, Philip S. Yu, Julian McAuley, and Caiming Xiong. 2021. Contrastive self-supervised sequential recommendation with robust augmentation. arXiv preprint arXiv:2108.06479 (2021).
[26]
Yingtao Luo, Qiang Liu, and Zhaocheng Liu. 2021. STAN: Spatio-temporal attention network for next location recommendation. In Proceedings of the Web Conference 2021 (WWW’21). ACM, New York, NY, 2177–2185.
[27]
Wesley Mathew, Ruben Raposo, and Bruno Martins. 2012. Predicting future locations with hidden Markov models. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing. ACM, New York, NY, 911–918.
[28]
Yifang Qin, Yifan Wang, Fang Sun, Wei Ju, Xuyang Hou, Zhe Wang, Jia Cheng, Jun Lei, and Ming Zhang. 2023. DisenPOI: Disentangling sequential and geographical influence for point-of-interest recommendation. In Proceedings of the 16th ACM International Conference on Web Search and Data Mining. 508–516.
[29]
Ruihong Qiu, Zi Huang, Hongzhi Yin, and Zijian Wang. 2022. Contrastive learning for representation degeneration problem in sequential recommendation. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining. 813–823.
[30]
Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the 19th International Conference on World Wide Web (WWW’10). ACM, New York, NY, 811–820.
[31]
Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. 2019. AutoInt: Automatic feature interaction learning via self-attentive neural networks. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1161–1170.
[32]
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15, 1 (2014), 1929–1958.
[33]
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from Transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1441–1450.
[34]
Ke Sun, Tieyun Qian, Tong Chen, Yile Liang, Quoc Viet Hung Nguyen, and Hongzhi Yin. 2020. Where to go next: Modeling long-and short-term user preferences for point-of-interest recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence. 214–221.
[35]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 6000–6010.
[36]
Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2021. Self-supervised graph learning for recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 726–735.
[37]
Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019. Session-based recommendation with graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence. 346–353.
[38]
Yuxia Wu, Ke Li, Guoshuai Zhao, and Xueming Qian. 2022. Personalized long- and short-term preference learning for next POI recommendation. IEEE Transactions on Knowledge and Data Engineering 34, 4 (2022), 1944–1957.
[39]
Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Jiandong Zhang, Bolin Ding, and Bin Cui. 2022. Contrastive learning for sequential recommendation. In Proceedings of the 2022 IEEE 38th International Conference on Data Engineering (ICDE’22). IEEE, Los Alamitos, CA, 1259–1273.
[40]
Dingqi Yang, Daqing Zhang, Vincent W. Zheng, and Zhiyong Yu. 2014. Modeling user activity preference by leveraging user spatial temporal characteristics in LBSNs. IEEE Transactions on Systems, Man, and Cybernetics: Systems 45, 1 (2014), 129–142.
[41]
Di Yao, Chao Zhang, Jianhui Huang, and Jingping Bi. 2017. SERM: A recurrent model for next location prediction in semantic trajectories. In Proceedings of the 2017 ACM Conference on Information and Knowledge Management. ACM, New York, NY, 2411–2414.
[42]
Haochao Ying, Fuzhen Zhuang, Fuzheng Zhang, Yanchi Liu, Guandong Xu, Xing Xie, Hui Xiong, and Jian Wu. 2018. Sequential recommender system based on hierarchical attention network. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. 3926–3932.
[43]
Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Jundong Li, and Zi Huang. 2022. Self-supervised learning for recommender systems: A survey. arXiv preprint arXiv:2203.15876 (2022).
[44]
Zeping Yu, Jianxun Lian, Ahmad Mahmoody, Gongshen Liu, and Xing Xie. 2019. Adaptive user modeling with long and short-term preferences for personalized recommendation. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 4213–4219.
[45]
Pengpeng Zhao, Anjing Luo, Yanchi Liu, Jiajie Xu, Zhixu Li, Fuzhen Zhuang, Victor S. Sheng, and Xiaofang Zhou. 2022. Where to go next: A spatio-temporal gated network for next POI recommendation. IEEE Transactions on Knowledge and Data Engineering 34, 5 (2022), 2512–2524.
[46]
Yu Zheng, Chen Gao, Jianxin Chang, Yanan Niu, Yang Song, Depeng Jin, and Yong Li. 2022. Disentangling long and short-term interests for recommendation. In Proceedings of the ACM Web Conference 2022. ACM, New York, NY, 2256–2267.
[47]
Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020. S3-Rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management. ACM, New York, NY, 1893–1902.

Cited By

View all
  • (2025)Learning Road Network Index Structure for Efficient Map MatchingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.348519537:1(423-437)Online publication date: 1-Jan-2025
  • (2025)Let long-term interests talk: An disentangled learning model for recommendation based on short-term interests generationInformation Processing & Management10.1016/j.ipm.2024.10399762:2(103997)Online publication date: Mar-2025
  • (2025)A user preference knowledge graph incorporating spatio-temporal transfer features for next POI recommendationApplied Intelligence10.1007/s10489-025-06290-y55:6Online publication date: 1-Apr-2025
  • Show More Cited By

Index Terms

  1. Modeling Long- and Short-Term User Preferences via Self-Supervised Learning for Next POI Recommendation

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Knowledge Discovery from Data
      ACM Transactions on Knowledge Discovery from Data  Volume 17, Issue 9
      November 2023
      373 pages
      ISSN:1556-4681
      EISSN:1556-472X
      DOI:10.1145/3604532
      Issue’s Table of Contents

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 15 June 2023
      Online AM: 12 May 2023
      Accepted: 01 May 2023
      Revised: 20 March 2023
      Received: 04 October 2022
      Published in TKDD Volume 17, Issue 9

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Next POI recommendation
      2. spatio-temporal context
      3. self-supervised learning
      4. attention architecture
      5. pre-training

      Qualifiers

      • Research-article

      Funding Sources

      • National Key R&D Program of China
      • Natural Science Foundation of China

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)518
      • Downloads (Last 6 weeks)34
      Reflects downloads up to 22 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2025)Learning Road Network Index Structure for Efficient Map MatchingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.348519537:1(423-437)Online publication date: 1-Jan-2025
      • (2025)Let long-term interests talk: An disentangled learning model for recommendation based on short-term interests generationInformation Processing & Management10.1016/j.ipm.2024.10399762:2(103997)Online publication date: Mar-2025
      • (2025)A user preference knowledge graph incorporating spatio-temporal transfer features for next POI recommendationApplied Intelligence10.1007/s10489-025-06290-y55:6Online publication date: 1-Apr-2025
      • (2024)AGENDA: Predicting Trip Purposes with A New Graph Embedding Network and Active Domain AdaptationACM Transactions on Knowledge Discovery from Data10.1145/367702018:8(1-25)Online publication date: 8-Jul-2024
      • (2024)Spatial-Temporal Interval Aware Individual Future Trajectory PredictionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.333292936:10(5374-5387)Online publication date: 1-Oct-2024
      • (2024)LEAP: Lifelong Learning Edge-Cloud Adaptive Fused Framework for Mobility Prediction2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10825926(6707-6716)Online publication date: 15-Dec-2024
      • (2024)POI recommendation for random groups based on cooperative graph neural networksInformation Processing & Management10.1016/j.ipm.2024.10367661:3(103676)Online publication date: May-2024
      • (2024)Point-of-interest recommendation based on LBSN with multi-aspect fusion of social and individual featuresNeural Computing and Applications10.1007/s00521-024-09711-036:20(12163-12184)Online publication date: 1-Jul-2024
      • (2023)Quality-Guaranteed and Cost-Effective Population Health Profiling: A Deep Active Learning ApproachACM Transactions on Computing for Healthcare10.1145/36171794:4(1-19)Online publication date: 13-Oct-2023

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      Full Text

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media