skip to main content
research-article

Pre-Training Across Different Cities for Next POI Recommendation

Published: 10 October 2023 Publication History

Abstract

The Point-of-Interest (POI) transition behaviors could hold absolute sparsity and relative sparsity very differently for different cities. Hence, it is intuitive to transfer knowledge across cities to alleviate those data sparsity and imbalance problems for next POI recommendation. Recently, pre-training over a large-scale dataset has achieved great success in many relevant fields, like computer vision and natural language processing. By devising various self-supervised objectives, pre-training models can produce more robust representations for downstream tasks. However, it is not trivial to directly adopt such existing pre-training techniques for next POI recommendation, due to the lacking of common semantic objects (users or items) across different cities. Thus in this paper, we tackle such a new research problem of pre-training across different cities for next POI recommendation. Specifically, to overcome the key challenge that different cities do not share any common object, we propose a novel pre-training model named CATUS, by transferring the category-level universal transition knowledge over different cities. Firstly, we build two self-supervised objectives in CATUS: next category prediction and next POI prediction, to obtain the universal transition-knowledge across different cities and POIs. Then, we design a category-transition oriented sampler on the data level and an implicit and explicit transfer strategy on the encoder level to enhance this transfer process. At the fine-tuning stage, we propose a distance oriented sampler to better align the POI representations into the local context of each city. Extensive experiments on two large datasets consisting of four cities demonstrate the superiority of our proposed CATUS over the state-of-the-art alternatives. The code and datasets are available at https://github.com/NLPWM-WHU/CATUS.

References

[1]
Buru Chang, Yonggyu Park, Donghyeon Park, Seongsoon Kim, and Jaewoo Kang. 2018. Content-aware hierarchical point-of-interest embedding model for successive POI recommendation. In IJCAI. 3301–3307.
[2]
Yiqi Chen, Tieyun Qian, Huan Liu, and Ke Sun. 2018. “Bridge” enhanced signed directed network embedding. In CIKM. 773–782.
[3]
Yudong Chen, Xin Wang, Miao Fan, Jizhou Huang, Shengwen Yang, and Wenwu Zhu. 2021. Curriculum meta-learning for next POI recommendation. In KDD. 2692–2702.
[4]
Chen Cheng, Haiqin Yang, Michael R. Lyu, and Irwin King. 2013. Where you like to go next: Successive point-of-interest recommendation. In IJCAI. 2605–2611.
[5]
Mingyue Cheng, Fajie Yuan, Qi Liu, Xin Xin, and Enhong Chen. 2021. Learning transferable user representations with sequential behaviors via contrastive pre-training. In ICDM. 51–60.
[6]
Qiang Cui, Chenrui Zhang, Yafeng Zhang, Jinpeng Wang, and Mingchen Cai. 2021. ST-PIL: Spatial-temporal periodic interest learning for next point-of-interest recommendation. In CIKM. 2960–2964.
[7]
Yue Cui, Hao Sun, Yan Zhao, Hongzhi Yin, and Kai Zheng. 2021. Sequential-knowledge-aware next POI recommendation: A meta-learning approach. TOIS 40, 2 (2021), 1–22.
[8]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
[9]
Jie Feng, Yong Li, Chao Zhang, Funing Sun, Fanchao Meng, Ang Guo, and Depeng Jin. 2018. DeepMove: Predicting human mobility with attentional recurrent networks. In WWW. 1459–1468.
[10]
Shanshan Feng, Xutao Li, Yifeng Zeng, Gao Cong, Yeow Meng Chee, and Quan Yuan. 2015. Personalized ranking metric embedding for next new POI recommendation. In IJCAI. 2069–2075.
[11]
Qing Guo, Zhu Sun, Jie Zhang, and Yin-Leng Theng. 2020. An attentional recurrent neural network for personalized next location recommendation. In AAAI. 83–90.
[12]
Bowen Hao, Hongzhi Yin, Jing Zhang, Cuiping Li, and Hong Chen. 2021. A multi-strategy based pre-training method for cold-start recommendation. arXiv preprint arXiv:2112.02275 (2021).
[13]
Bowen Hao, Jing Zhang, Hongzhi Yin, Cuiping Li, and Hong Chen. 2021. Pre-training graph neural networks for cold-start users and items representation. In WSDM. 265–273.
[14]
Jing He, Xin Li, and Lejian Liao. 2017. Category-aware next point-of-interest recommendation via listwise Bayesian personalized ranking. In IJCAI. 1837–1843.
[15]
Jing He, Xin Li, Lejian Liao, Dandan Song, and William Cheung. 2016. Inferring a personalized next point-of-interest recommendation model with latent behavior patterns. In AAAI. 137–143.
[16]
Liwei Huang, Yutao Ma, Yanbo Liu, and Keqing He. 2020. DAN-SNR: A deep attentive network for social-aware next point-of-interest recommendation. ACM Transactions on Internet Technology 21, 1 (2020), 1–27.
[17]
Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In ICDM. 197–206.
[18]
Minseok Kim, Hwanjun Song, Doyoung Kim, Kijung Shin, and Jae-Gil Lee. 2021. PREMERE: Meta-reweighting via self-ensembling for point-of-interest recommendation. In AAAI. 4164–4171.
[19]
Dejiang Kong and Fei Wu. 2018. HST-LSTM: A hierarchical spatial-temporal long-short term memory network for location prediction. In IJCAI. 2341–2347.
[20]
Ranzhen Li, Yanyan Shen, and Yanmin Zhu. 2018. Next point-of-interest recommendation with temporal and multi-level context attention. In ICDM. 1110–1115.
[21]
Yang Li, Tong Chen, Hongzhi Yin, and Zi Huang. 2021. Discovering collaborative signals for next POI recommendation with iterative Seq2Graph augmentation. In IJCAI. 1491–1497.
[22]
Zeyu Li, Wei Cheng, Haiqi Xiao, Wenchao Yu, Haifeng Chen, and Wei Wang. 2021. You are what and where you are: Graph enhanced attention network for explainable POI recommendation. In CIKM. 3945–3954.
[23]
Defu Lian, Yongji Wu, Yong Ge, Xing Xie, and Enhong Chen. 2020. Geography-aware sequential location recommendation. In KDD. 2009–2019.
[24]
Nicholas Lim, Bryan Hooi, See-Kiong Ng, Yong Liang Goh, Renrong Weng, and Rui Tan. 2022. Hierarchical multi-task graph recurrent network for next POI recommendation. (2022), 1133–1143.
[25]
Nicholas Lim, Bryan Hooi, See-Kiong Ng, Xueou Wang, Yong Liang Goh, Renrong Weng, and Jagannadan Varadarajan. 2020. STP-UDGAT: Spatial-temporal-preference user dimensional graph attention network for next POI recommendation. In CIKM. 845–854.
[26]
Yan Lin, Huaiyu Wan, Shengnan Guo, and Youfang Lin. 2021. Pre-training context and time aware location embeddings from spatial-temporal trajectories for user next location prediction. In AAAI. 4241–4248.
[27]
Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. 2016. Predicting the next location: A recurrent model with spatial and temporal contexts. In AAAI. 194–200.
[28]
Yong Liu, Susen Yang, Chenyi Lei, Guoxin Wang, Haihong Tang, Juyong Zhang, Aixin Sun, and Chunyan Miao. 2021. Pre-training graph transformer with multimodal side information for recommendation. In MM. 2853–2861.
[29]
Yingtao Luo, Qiang Liu, and Zhaocheng Liu. 2021. STAN: Spatio-temporal attention network for next location recommendation. In WWW. 2177–2185.
[30]
Xuan Ma, Tieyun Qian, Yile Liang, Ke Sun, Hang Yun, and Mi Zhang. 2022. Enhancing graph convolution network for novel recommendation. In DASFAA. Springer, 69–84.
[31]
Jarana Manotumruksa, Craig Macdonald, and Iadh Ounis. 2018. A contextual attention recurrent architecture for context-aware venue recommendation. In SIGIR. 555–564.
[32]
Tieyun Qian, Yile Liang, Qing Li, Xuan Ma, Ke Sun, and Zhiyong Peng. 2022. Intent disentanglement and feature self-supervision for novel recommendation. IEEE Transactions on Knowledge and Data Engineering (2022).
[33]
Tieyun Qian, Yile Liang, Qing Li, and Hui Xiong. 2020. Attribute graph neural networks for strict cold start recommendation. TKDE (2020).
[34]
Xuan Rao, Lisi Chen, Yong Liu, Shuo Shang, Bin Yao, and Peng Han. 2022. Graph-flashback network for next location recommendation. In KDD. 1463–1471.
[35]
Huimin Sun, Jiajie Xu, Kai Zheng, Pengpeng Zhao, Pingfu Chao, and Xiaofang Zhou. 2021. MFNP: A meta-optimized model for few-shot next POI recommendation. In IJCAI. 3017–3023.
[36]
Ke Sun and Tieyun Qian. 2018. Exploiting user and item attributes for sequential recommendation. In ICONIP. Springer, 370–380.
[37]
Ke Sun, Tieyun Qian, Tong Chen, Yile Liang, Quoc Viet Hung Nguyen, and Hongzhi Yin. 2020. Where to go next: Modeling long-and short-term user preferences for point-of-interest recommendation. In AAAI. 214–221.
[38]
Ke Sun, Tieyun Qian, Xu Chen, and Ming Zhong. 2021. Context-aware seq2seq translation model for sequential recommendation. Information Sciences 581 (2021), 60–72.
[39]
Ke Sun, Tieyun Qian, Hongzhi Yin, Tong Chen, Yiqi Chen, and Ling Chen. 2019. What can history tell us?. In CIKM. 1593–1602.
[40]
Ke Sun, Tieyun Qian, Ming Zhong, and Xuhui Li. 2023. Towards more effective encoders in pre-training for sequential recommendation. World Wide Web (2023), 1–32.
[41]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS. 5998–6008.
[42]
Huaiyu Wan, Yan Lin, Shengnan Guo, and Youfang Lin. 2021. Pre-training time-aware location embeddings from spatial-temporal trajectories. TKDE (2021).
[43]
Chen Wang, Yueqing Liang, Zhiwei Liu, Tao Zhang, and Philip S. Yu. 2021. Pre-training graph neural network for cross domain recommendation. arXiv preprint arXiv:2111.08268 (2021).
[44]
Lingzhi Wang, Xingshan Zeng, Huang Hu, Kam-Fai Wong, and Daxin Jiang. 2021. Re-entry prediction for online conversations via self-supervised learning. In Findings of EMNLP. 2127–2137.
[45]
Peng Wang, Jiang Xu, Chunyi Liu, Hao Feng, Zang Li, and Jieping Ye. 2020. Masked-field pre-training for user intent prediction. In CIKM. 2789–2796.
[46]
Xiaolin Wang, Guohao Sun, Xiu Fang, Jian Yang, and Shoujin Wang. 2022. Modeling spatio-temporal neighbourhood for personalized point-of-interest recommendation. In IJCAI. 3530–3536.
[47]
Zhaobo Wang, Yanmin Zhu, Haobing Liu, and Chunyang Wang. 2022. Learning graph-based disentangled representations for next POI recommendation. In SIGIR. 1154–1163.
[48]
Chuhan Wu, Fangzhao Wu, Tao Qi, Jianxun Lian, Yongfeng Huang, and Xing Xie. 2020. PTUM: Pre-training user model from unlabeled user behaviors via self-supervision. arXiv preprint arXiv:2010.01494 (2020).
[49]
Chuhan Wu, Fangzhao Wu, Yang Yu, Tao Qi, Yongfeng Huang, and Xing Xie. 2021. UserBERT: Contrastive user model pre-training. arXiv preprint arXiv:2109.01274 (2021).
[50]
Chaojun Xiao, Ruobing Xie, Yuan Yao, Zhiyuan Liu, Maosong Sun, Xu Zhang, and Leyu Lin. 2021. UPRec: User-aware pre-training for recommender systems. arXiv preprint arXiv:2102.10989 (2021).
[51]
Carl Yang, Lanxiao Bai, Chao Zhang, Quan Yuan, and Jiawei Han. 2017. Bridging collaborative filtering and semi-supervised learning: A neural approach for POI recommendation. In KDD. 1245–1254.
[52]
Song Yang, Jiamou Liu, and Kaiqi Zhao. 2022. GETNext: Trajectory flow map enhanced transformer for next POI recommendation. In SIGIR. 1144–1153.
[53]
Di Yao, Chao Zhang, Jianhui Huang, and Jingping Bi. 2017. SERM: A recurrent model for next location prediction in semantic trajectories. In CIKM. 2411–2414.
[54]
Mao Ye, Peifeng Yin, Wang-Chien Lee, and Dik-Lun Lee. 2011. Exploiting geographical influence for collaborative point-of-interest recommendation. In SIGIR. 325–334.
[55]
Fuqiang Yu, Lizhen Cui, Wei Guo, Xudong Lu, Qingzhong Li, and Hua Lu. 2020. A category-aware deep model for successive POI recommendation on sparse check-in data. In WWW. 1264–1274.
[56]
Xu Yuan, Hongshen Chen, Yonghao Song, Xiaofang Zhao, Zhuoye Ding, Zhen He, and Bo Long. 2021. Improving sequential recommendation consistency with self-supervised imitation. In IJCAI. 3321–3327.
[57]
Lu Zhang, Zhu Sun, Ziqing Wu, Jie Zhang, Yew Soon Ong, and Xinghua Qu. [n. d.]. Next point-of-interest recommendation with inferring multi-step future preferences.
[58]
Lu Zhang, Zhu Sun, Jie Zhang, Yu Lei, Chen Li, Ziqing Wu, Horst Kloeden, and Felix Klanner. 2020. An interactive multi-task learning framework for next POI recommendation with uncertain check-ins. In IJCAI. 3551–3557.
[59]
Mingwei Zhang, Yang Yang, Rizwan Abbas, Ke Deng, Jianxin Li, and Bin Zhang. 2021. SNPR: A serendipity-oriented next POI recommendation model. In CIKM. 2568–2577.
[60]
Pengpeng Zhao, Haifeng Zhu, Yanchi Liu, Jiajie Xu, Zhixu Li, Fuzhen Zhuang, Victor S. Sheng, and Xiaofang Zhou. 2019. Where to go next: A spatio-temporal gated network for next POI recommendation. In AAAI. 5877–5884.
[61]
Shenglin Zhao, Tong Zhao, Haiqin Yang, Michael R. Lyu, and Irwin King. 2016. STELLAR: Spatial-temporal latent ranking for successive point-of-interest recommendation. In AAAI. 315–322.
[62]
Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In CIKM. 1893–1902.

Cited By

View all
  • (2024)Urban Foundation Models: A SurveyProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671453(6633-6643)Online publication date: 25-Aug-2024
  • (2024)CrossPred: A Cross-City Mobility Prediction Framework for Long-Distance Travelers via POI Feature MatchingProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679893(4148-4152)Online publication date: 21-Oct-2024
  • (2024)ImNextKnowledge-Based Systems10.1016/j.knosys.2024.111674293:COnline publication date: 7-Jun-2024

Index Terms

  1. Pre-Training Across Different Cities for Next POI Recommendation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on the Web
    ACM Transactions on the Web  Volume 17, Issue 4
    November 2023
    331 pages
    ISSN:1559-1131
    EISSN:1559-114X
    DOI:10.1145/3608910
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 10 October 2023
    Online AM: 20 June 2023
    Accepted: 17 June 2023
    Revised: 19 April 2023
    Received: 29 December 2022
    Published in TWEB Volume 17, Issue 4

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Sequential POI recommendation
    2. pre-training
    3. sparsity

    Qualifiers

    • Research-article

    Funding Sources

    • NSFC

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)447
    • Downloads (Last 6 weeks)33
    Reflects downloads up to 17 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Urban Foundation Models: A SurveyProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671453(6633-6643)Online publication date: 25-Aug-2024
    • (2024)CrossPred: A Cross-City Mobility Prediction Framework for Long-Distance Travelers via POI Feature MatchingProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679893(4148-4152)Online publication date: 21-Oct-2024
    • (2024)ImNextKnowledge-Based Systems10.1016/j.knosys.2024.111674293:COnline publication date: 7-Jun-2024

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media