Skip to main content
SpringerLink
Log in

Hybrid graph convolutional networks with multi-head attention for location recommendation

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Recommending yet-unvisited points of interest (POIs) which may be of interest to users is one of the fundamental applications in location-based social networks. It mainly replies on the understanding of users, POIs, and their interactions. Previous studies either develop matrix factorization-based approaches or utilize deep learning frameworks to learn better representation of users and POIs in order to estimate users’ latent preference. However, most of existing methods still confront the challenges like in traditional recommender systems, such as data sparsity and cold-start. In particular, they have difficulties in fully utilizing rich semantic information, such as social influence, geographical constraints and interactions between users and POIs. To fill this research gap, we propose a new recommendation framework – Hybrid Graph convolutional networks with Multi-head Attention for POI recommendation (HGMAP). HGMAP constructs a spatial graph based on the geographical distance between pairs of POIs and leverages Graph Convolutional Networks (GCNs) to express the high-order connectivity among POIs, which not only incorporates the spatial constraints but also provides an effective way to alleviate the sparse check-in problem. In addition, HGMAP exploits the user social relationship with another GCN and differentiates user preference over different aspects of POIs with a multi-head attention mechanism. We conducted extensive experiments on three public datasets and the results demonstrate that HGMAP significantly improves the recommendation performance over several state-of-the-art models, for example, up to approximately 4.8% and 7% for Precision@10 and Recall@10, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://www.yelp.com/dataset/challenge

References

  1. Altaf, B., Yu, L., Zhang, X.: Spatio-Temporal Attention Based Recurrent Neural Network for Next Location Prediction. In: IEEE International Conference on Big Data, Big Data 2018, Seattle, pp. 937–942 (2018)

  2. Bahdanau, D., Cho, K., Bengio, Y.: Neural Machine Translation by Jointly Learning to Align and Translate. In: ICLR (2015)

  3. Chen, J., Zhang, H., He, X., Nie, L., Liu, W., Chua, T. S.: Attentive collaborative filtering: Multimedia recommendation with item-and component-level attention. In: Proceedings of the International conference on Research and development in information retrieval (SIGIR), pp. 335–344 (2017)

  4. Chen, X., Zhou, F., Zhang, K., Trajcevski, G., Zhong, T., Zhang, F.: Information Diffusion Prediction via Recurrent Cascades Convolution. In: 2019 IEEE 35Th International Conference on Data Engineering (ICDE), pp. 770–781. IEEE (2019)

  5. Cheng, C., Yang, H., King, I., Lyu, M. R.: Fused matrix factorization with geographical and social influence in location-based social networks. In: Proceedings of the AAAI International Conference on Artificial Intelligence (2012)

  6. Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In: Advances in Neural Information Processing Systems (NIPS), pp. 3844–3852 (2016)

  7. Devlin, J., Chang, M. W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186 (2019)

  8. Eom, C. S., Lee, C. C., Lee, W., Leung, C. K.: Effective privacy preserving data publishing by vectorization. Information Sciences (2019)

  9. Fan, S., Zhu, J., Han, X., Shi, C., Hu, L., Ma, B., Li, Y.: Metapath-guided heterogeneous graph neural network for intent recommendation. In: Proceedings of the International Conference on Knowledge Discovery & Data Mining (SIGKDD), pp. 2478–2486. ACM (2019)

  10. Gao, Q., Zhou, F., Zhang, K., Trajcevski, G., Luo, X., Zhang, F.: Identifying human mobility via trajectory embeddings. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp. 1689–1695 (2017)

  11. Gao, Q., Zhou, F., Trajcevski, G., Zhang, K., Ting, Z., Zhang, F.: Predicting human mobility via variational attention. In: Proceedings of the International Conference on World Wide Web (WWW), pp. 2750–2756. ACM (2019)

  12. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative Adversarial Nets. In: Advances in Neural Information Processing Systems (NIPS), pp. 2672–2680 (2014)

  13. Hang, M., Pytlarz, I., Neville, J.: Exploring student check-in behavior for improved point-of-interest prediction. In: Proceedings of the International Conference on Knowledge Discovery & Data Mining (SIGKDD), pp. 321–330. ACM (2018)

  14. Hao, M., Chao, L., King, I., Lyu, M. R.: Probabilistic factor models for web site recommendation. In: Proceedings of the International conference on Research and development in information retrieval (SIGIR), pp. 265–274. ACM (2011)

  15. Hosseini, S., Yin, H., Zhou, X., Sadiq, S., Kangavari, M. R., Cheung, N. M.: Leveraging multi-aspect time-related influence in location recommendation. World Wide Web 22(3), 1001–1028 (2019)

    Article  Google Scholar 

  16. Hu, Y., Koren, Y., Volinsky, C.: Collaborative filtering for implicit feedback datasets. In: Proceedings of the International Conference on Data Mining (ICDM), pp. 263–272. IEEE (2008)

  17. Kingma, D. P., Ba, J.: Adam: A method for stochastic optimization. arXiv:1412.6980 (2014)

  18. Kipf, T. N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: Proceedings of the International Conference on Learning Representations (ICLR) (2017)

  19. Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. J. Comput. 48(8), 30–37 (2009)

    Google Scholar 

  20. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: Albert: a Lite Bert for Self-Supervised Learning of Language Representations. In: International Conference on Learning Representations (2019)

  21. Lee, W., Song, K., Moon, I. C.: Augmented variational autoencoders for collaborative filtering with auxiliary information. In: Proceedings of the International Conference on Information and Knowledge Management (CIKM), pp. 1139–1148. ACM (2017)

  22. Li, X., Cong, G., Li, X. L., Pham, T. A. N., Krishnaswamy, S.: Rank-geofm: a ranking based geographical factorization method for point of interest recommendation. In: Proceedings of the International conference on Research and development in information retrieval (SIGIR), pp. 433–442 (2015)

  23. Li, X., Cong, G., Li, X. L., Pham, T. A. N., Krishnaswamy, S.: Rank-geofm: a ranking based geographical factorization method for point of interest recommendation. In: Proceedings of the International conference on Research and development in information retrieval (SIGIR), pp. 433–442. ACM (2015)

  24. Li, H., Ge, Y., Hong, R., Zhu, H.: Point-of-interest recommendations: Learning potential check-ins from friends. In: Proceedings of the International Conference on Knowledge Discovery & Data Mining (SIGKDD), pp. 975–984. ACM (2016)

  25. Li, X., She, J.: Collaborative variational autoencoder for recommender systems. In: Proceedings of the International Conference on Knowledge Discovery & Data Mining (SIGKDD), pp. 305–314. ACM (2017)

  26. Lian, D., Zhao, C., Xie, X., Sun, G., Chen, E., Rui, Y.: Geomf: joint geographical modeling and matrix factorization for point-of-interest recommendation. In: Proceedings of the International Conference on Knowledge Discovery & Data Mining (SIGKDD), pp. 831–840. ACM (2014)

  27. Liang, D., Krishnan, R. G., Hoffman, M. D., Jebara, T.: Variational autoencoders for collaborative filtering. In: Proceedings of the International Conference on World Wide Web (WWW), pp. 689–698. ACM (2018)

  28. Liu, B., Fu, Y., Yao, Z., Xiong, H.: Learning geographical preferences for point-of-interest recommendation. In: Proceedings of the International Conference on Knowledge Discovery & Data Mining (SIGKDD), pp. 1043–1051. ACM (2013)

  29. Liu, Y., Wei, W., Sun, A., Miao, C.: Exploiting geographical neighborhood characteristics for location recommendation. In: Proceedings of the International Conference on Information and Knowledge Management (CIKM), pp. 739–748. ACM (2014)

  30. Liu, B., Xiong, H., Papadimitriou, S., Fu, Y., Yao, Z.: A general geographical probabilistic factor model for point of interest recommendation. IEEE Trans. Knowl. Data Eng. (TKDE) 27(5), 1167–1179 (2015)

    Article  Google Scholar 

  31. Liu, Y., Pham, T. A. N., Cong, G., Yuan, Q.: An experimental evaluation of point-of-interest recommendation in location-based social networks. Proc. VLDB Endowment 10(10), 1010–1021 (2017)

    Article  Google Scholar 

  32. Liu, W., Wang, Z. J., Yao, B., Yin, J.: Geo-alm: Poi recommendation by fusing geographical information and adversarial learning mechanism. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp. 1807–1813 (2019)

  33. Lu, Y. S., Shih, W. Y., Gau, H. Y., Chung, K. C., Huang, J. L.: On successive point-of-interest recommendation. World Wide Web 22(3), 1151–1173 (2019)

    Article  Google Scholar 

  34. Ma, C., Zhang, Y., Wang, Q., Liu, X.: Point-of-interest recommendation: Exploiting self-attentive autoencoders with neighbor-aware influence. In: Proceedings of the International Conference on Information and Knowledge Management (CIKM), pp. 697–706. ACM (2018)

  35. Ma, F., Gao, F., Sun, J., Zhou, H., Hussain, A.: Attention graph convolution network for image segmentation in big SAR imagery data. Remote. Sens. 11(21), 2586 (2019)

    Article  Google Scholar 

  36. Maaten, L.v.d., Hinton, G.: Visualizing data using t-sne. J. Mach. Learn. Res. (JMLR) 9, 2579–2605 (2008)

    MATH  Google Scholar 

  37. Manotumruksa, J., Macdonald, C., Ounis, I.: A contextual attention recurrent architecture for context-aware venue recommendation. In: Proceedings of the International conference on Research and development in information retrieval (SIGIR), pp. 555–564. ACM (2018)

  38. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., Dean, J.: Distributed Representations of Words and Phrases and Their Compositionality. In: Advances in Neural Information Processing Systems (NIPS), pp. 3111–3119 (2013)

  39. Monti, F., Bronstein, M., Bresson, X.: Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 3697–3707 (2017)

  40. Qian, T., Liu, B., Nguyen, Q. V. H., Yin, H.: Spatiotemporal representation learning for translation-based poi recommendation. ACM Trans. Inf. Syst. (TOIS) 37 (2), 18:1–18:24 (2019)

    Article  Google Scholar 

  41. Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: Bpr: Bayesian personalized ranking from implicit feedback. In: Proceedings of Internation conference on uncertainty in artificial intelligence (UAI), pp. 452–461. AUAI Press (2009)

  42. Salakhutdinov, R., Mnih, A.: Bayesian Probabilistic Matrix Factorization Using Markov Chain Monte Carlo. In: International Conference on Machine Learning (ICML), pp. 880–887 (2008)

  43. van den Berg, R., Kipf, T. N., Welling, M.: Graph convolutional matrix completion. arXiv:1706.02263v2 (2017)

  44. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., Polosukhin, I.: Attention is All You Need. In: Advances in Neural Information Processing Systems (NIPS), pp. 5998–6008 (2017)

  45. Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Liȯ, P., Bengio, Y.: Graph attention networks. In: Proceedings of the International Conference on Learning Representations (ICLR) (2018)

  46. Wang, W., Yin, H., Du, X., Nguyen, Q. V. H., Zhou, X.: TPM: a temporal personalized model for spatial item recommendation. ACM Trans. Intell. Syst. Technol. (TIST) 9(6), 61:1–61:25 (2018)

    Google Scholar 

  47. Wang, H., Zhang, F., Zhang, M., Leskovec, J., Zhao, M., Li, W.: Wang, Z.: Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems. In: Proceedings of the International Conference on Knowledge Discovery & Data Mining (SIGKDD), pp. 968–977. ACM (2019)

  48. Wang, H., Zhao, M., Xie, X., Li, W., Guo, M.: Knowledge graph convolutional networks for recommender systems. In: Proceedings of the International Conference on World Wide Web (WWW), pp. 3307–3313. ACM (2019)

  49. Wang, X., He, X., Cao, Y., Liu, M., Chua, T.: KGAT: knowledge graph attention network for recommendation. In: Proceedings of the International Conference on Knowledge Discovery & Data Mining (SIGKDD), pp. 950–958. ACM (2019)

  50. Wang, X., He, X., Wang, M., Feng, F., Chua, T.: Neural graph collaborative filtering. In: Proceedings of the International conference on Research and development in information retrieval (SIGIR), pp. 165–174. ACM (2019)

  51. Wu, F., Zhang, T., Souza, Jr., A.H.d., Fifty, C., Yu, T., Weinberger, K.Q.: Simplifying Graph Convolutional Networks. In: International Conference on Machine Learning (ICML), pp. 6861–6871 (2019)

  52. Yang, C., Bai, L., Zhang, C., Yuan, Q., Han, J.: Bridging collaborative filtering and semi-supervised learning: a neural approach for poi recommendation. In: Proceedings of the International Conference on Knowledge Discovery & Data Mining (SIGKDD), pp. 1245–1254. ACM (2017)

  53. Ye, M., Yin, P., Lee, W. C., Lee, D. L.: Exploiting geographical influence for collaborative point-of-interest recommendation. In: Proceedings of the International conference on Research and development in information retrieval (SIGIR), pp. 325–334. ACM (2011)

  54. Yin, H., Zhou, X., Cui, B., Wang, H., Zheng, K., Hung, N. Q. V.: Adapting to user interest drift for poi recommendation. IEEE Trans. Knowl. Data Eng. (TKDE) 28(10), 2566–2581 (2016)

    Article  Google Scholar 

  55. Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W. L., Leskovec, J.: Graph convolutional neural networks for web-scale recommender systems. In: Proceedings of the SIGKDD International Conference on Knowledge Discovery & Data Mining (SIGKDD), pp. 974–983. ACM (2018)

  56. Ying, H., Wu, J., Xu, G., Liu, Y., Liang, T., Zhang, X., Xiong, H.: Time-aware metric embedding with asymmetric projection for successive poi recommendation. World Wide Web 22(5), 2209–2224 (2019)

    Article  Google Scholar 

  57. Yuan, Q., Cong, G., Ma, Z., Sun, A., Thalmann, N. M.: Time-aware point-of-interest recommendation. In: Proceedings of the International conference on Research and development in information retrieval (SIGIR), pp. 363–372. ACM (2013)

  58. Zhang, J. D., Chow, C. Y.: Geosoca: Exploiting geographical, social and categorical correlations for point-of-interest recommendations. In: Proceedings of the International conference on Research and development in information retrieval (SIGIR), pp. 443–452. ACM (2015)

  59. Zhang, C., Kim, J.: Object Detection with Location-Aware Deformable Convolution and Backward Attention Filtering. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 9452–9461 (2019)

  60. Zhang, S., Yao, L., Sun, A., Tay, Y.: Deep learning based recommender system: A survey and new perspectives. ACM Comput. Surv. 52(1), 5:1–5:38 (2019)

    Google Scholar 

  61. Zhang, Z., Liu, Y., Zhang, Z., Shen, B.: Fused matrix factorization with multi-tag, social and geographical influences for poi recommendation. World Wide Web 22(3), 1135–1150 (2019)

    Article  Google Scholar 

  62. Zhang, Y., Feng, Y., Shang, J., Zhou, M., Qiang, B.: Attention-aware joint location constraint hashing for multi-label image retrieval. IEEE Access 8, 3294–3307 (2020)

    Article  Google Scholar 

  63. Zhao, S., Zhao, T., King, I., Lyu, M. R.: Geo-teaser: Geo-temporal sequential embedding rank for point-of-interest recommendation. In: Proceedings of the International Conference on World Wide Web (WWW), pp. 153–162. ACM (2017)

  64. Zhao, P., Zhu, H., Liu, Y., Xu, J., Li, Z., Zhuang, F., Sheng, V. S., Zhou, X.: Where to go next: a spatio-temporal gated network for next poi recommendation. In: Proceedings of the AAAI International Conference on Artificial Intelligence, pp. 5877–5884 (2019)

  65. Zhong, T., Wen, Z., Zhou, F., Trajcevski, G., Zhang, K.: Session-based recommendation via flow-based deep generative networks and bayesian inference. Neurocomputing (2020)

  66. Zhou, F., Gao, Q., Zhang, K., Trajcevski, G., Ting, Z., Zhang, F.: Trajectory-user linking via variational autoencoder. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp. 3212–3218 (2018)

  67. Zhou, F., Cao, C., Zhang, K., Trajcevski, G., Zhong, T., Geng, J.: Meta-gnn: on few-shot node classification in graph meta-learning. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 2357–2360 (2019)

  68. Zhou, F., Wen, Z., Trajcevski, G., Zhang, K., Zhong, T., Liu, F.: Disentangled Network Alignment with Matching Explainability. In: IEEE INFOCOM 2019-IEEE Conference on Computer Communications, pp. 1360–1368. IEEE (2019)

  69. Zhou, F., Wen, Z., Zhang, K., Trajcevski, G., Zhong, T.: Variational session-based recommendation using normalizing flows. In: Proceedings of the International Conference on World Wide Web (WWW), pp. 3476–3475. ACM (2019)

  70. Zhou, F., Yin, R., Trajcevski, G., Zhang, K., Wu, J., Khokhar, A.: Improving human mobility identification with trajectory augmentation. GeoInformatica (2019)

  71. Zhou, F., Yin, R., Zhang, K., Trajcevski, G., Zhong, T., Wu, J.: Adversarial point-of-interest recommendation. In: Proceedings of the International Conference on World Wide Web (WWW), pp. 3462–34618. ACM (2019)

  72. Zhou, F., Yue, X., Trajcevski, G., Zhong, T., Zhang, K.: Context-aware variational trajectory encoding and human mobility inference. In: Proceedings of the International Conference on World Wide Web (WWW), pp. 3469–3475. ACM (2019)

  73. Zhou, F., Mo, Y., Trajcevski, G., Zhang, K., Wu, J., Zhong, T.: Recommendation via collaborative autoregressive flows. Neural Networks (2020)

  74. Zhou, F., Yang, Q., Zhang, K., Trajcevski, G., Zhong, T., Khokhar, A.: Reinforced spatio-temporal attentive graph neural networks for traffic forecasting. IEEE Internet of Things Journal (2020)

Download references

Acknowledgments

This work was supported by National Natural Science Foundation of China (Grant No.61602097 and No.61472064), NSF grant CNS 1646107.

Author information

Authors and Affiliations

  1. School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, 610054, China

    Ting Zhong, Shengming Zhang, Fan Zhou & Jin Wu

  2. Department of Decision, Operations & Information Technologies, University of Maryland, College Park, MD, 20742, USA

    Kunpeng Zhang

  3. Department of Electrical and Computer Engineering, Iowa State University, Ames, Iowa, 50011, USA

    Goce Trajcevski

Authors
  1. Ting Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shengming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kunpeng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Goce Trajcevski
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fan Zhou.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhong, T., Zhang, S., Zhou, F. et al. Hybrid graph convolutional networks with multi-head attention for location recommendation. World Wide Web 23, 3125–3151 (2020). https://doi.org/10.1007/s11280-020-00824-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-020-00824-9

Keywords

Search

Navigation