skip to main content
10.1145/3394486.3403128acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

Incremental Mobile User Profiling: Reinforcement Learning with Spatial Knowledge Graph for Modeling Event Streams

Published: 20 August 2020 Publication History

Abstract

We study the integration of reinforcement learning and spatial knowledge graph for incremental mobile user profiling, which aims to map mobile users to dynamically-updated profile vectors by incremental learning from a mixed-user event stream. After exploring many profiling methods, we identify a new imitation based criteria to better evaluate and optimize profiling accuracy. Considering the objective of teaching an autonomous agent to imitate a mobile user to plan next-visit based on the user's profile, the user profile is the most accurate when the agent can perfectly mimic the activity patterns of the user. We propose to formulate the problem into a reinforcement learning task, where an agent is a next-visit planner, an action is a POI that a user will visit next, and the state of environment is a fused representation of a user and spatial entities (e.g., POIs, activity types, functional zones). An event that a user takes an action to visit a POI, will change the environment, resulting into a new state of user profiles and spatial entities, which helps the agent to predict next visit more accurately. After analyzing such interactions among events, users, and spatial entities, we identify (1)semantic connectivity among spatial entities, and, thus, introduce a spatial Knowledge Graph (KG) to characterize the semantics of user visits over connected locations, activities, and zones. Besides, we identify (2) mutual influence between users and the spatial KG, and, thus, develop a mutual-updating strategy between users and the spatial KG, mixed with temporal context, to quantify the state representation that evolves over time. Along these lines, we develop a reinforcement learning framework integrated with spatial KG. The proposed framework can achieve incremental learning in multi-user profiling given a mixed-user event stream. Finally, we apply our approach to human mobility activity prediction and present extensive experiments to demonstrate improved performances.

References

[1]
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Advances in neural information processing systems. 2787--2795.
[2]
Sungwoon Choi, Heonseok Ha, Uiwon Hwang, Chanju Kim, Jung-Woo Ha, and Sungroh Yoon. 2018. Reinforcement learning based recommender system using biclustering technique. arXiv preprint arXiv:1801.05532 (2018).
[3]
Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems. 191--198.
[4]
Daniela Godoy and Analia Amandi. 2005. User profiling in personal information agents: a survey. The Knowledge Engineering Review, Vol. 20, 4 (2005), 329--361.
[5]
Jean-Benoit Griesner, Talel Abdessalem, and Hubert Naacke. 2015. POI recommendation: Towards fused matrix factorization with geographical and temporal influences. In Proceedings of the 9th ACM Conference on Recommender Systems. 301--304.
[6]
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015).
[7]
Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 687--696.
[8]
Maciej Kula. 2017. Spotlight. https://github.com/maciejkula/spotlight.
[9]
Defu Lian, Cong Zhao, Xing Xie, Guangzhong Sun, Enhong Chen, and Yong Rui. 2014. GeoMF: joint geographical modeling and matrix factorization for point-of-interest recommendation. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 831--840.
[10]
Long-Ji Lin. 1993. Reinforcement learning for robots using neural networks. Technical Report. Carnegie-Mellon Univ Pittsburgh PA School of Computer Science.
[11]
Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Twenty-ninth AAAI conference on artificial intelligence.
[12]
Kunpeng Liu, Pengyang Wang, Jiawei Zhang, Yanjie Fu, and Sajal K Das. 2018. Modeling the Interaction Coupling of Multi-View Spatiotemporal Contexts for Destination Prediction. In Proceedings of the 2018 SIAM International Conference on Data Mining. SIAM, 171--179.
[13]
Andriy Mnih and Russ R Salakhutdinov. 2008. Probabilistic matrix factorization. In Advances in neural information processing systems. 1257--1264.
[14]
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013).
[15]
Isshu Munemasa, Yuta Tomomatsu, Kunioki Hayashi, and Tomohiro Takagi. 2018. Deep reinforcement learning for recommender systems. In 2018 international conference on information and communications technology (icoiact). IEEE, 226--233.
[16]
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. 2016. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499 (2016).
[17]
Pengyang Wang, Yanjie Fu, Hui Xiong, and Xiaolin Li. 2019. Adversarial Substructured Representation Learning for Mobile User Profiling. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 130--138.
[18]
Pengyang Wang, Yanjie Fu, Yuanchun Zhou, Kunpeng Liu, Xiaolin Li, and Kien Hua. 2020. Exploiting Mutual Information for Substructure-aware Graph Representation Learning. In Proceedings of the 29th International Joint Conference on Artificial Intelligence. AAAI Press.
[19]
Pengyang Wang, Jiawei Zhang, Guannan Liu, Yanjie Fu, and Charu Aggarwal. 2018. Ensemble-Spotting: Ranking Urban Vibrancy via POI Embedding with Multi-view Spatial Graphs. In Proceedings of the 2018 SIAM International Conference on Data Mining. SIAM, 351--359.
[20]
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Twenty-Eighth AAAI conference on artificial intelligence.
[21]
Carl Yang, Lanxiao Bai, Chao Zhang, Quan Yuan, and Jiawei Han. 2017. Bridging collaborative filtering and semi-supervised learning: a neural approach for poi recommendation. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1245--1254.
[22]
Dingqi Yang, Daqing Zhang, Vincent W Zheng, and Zhiyong Yu. 2014. Modeling user activity preference by leveraging user spatial temporal characteristics in LBSNs. IEEE Transactions on Systems, Man, and Cybernetics: Systems, Vol. 45, 1 (2014), 129--142.
[23]
Hongzhi Yin, Weiqing Wang, Hao Wang, Ling Chen, and Xiaofang Zhou. 2017. Spatial-aware hierarchical collaborative deep learning for POI recommendation. IEEE Transactions on Knowledge and Data Engineering, Vol. 29, 11 (2017), 2537--2551.
[24]
Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys (CSUR), Vol. 52, 1 (2019), 1--38.
[25]
Xiangyu Zhao, Liang Zhang, Zhuoye Ding, Long Xia, Jiliang Tang, and Dawei Yin. 2018. Recommendations with negative feedback via pairwise deep reinforcement learning. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1040--1048.
[26]
Guanjie Zheng, Fuzheng Zhang, Zihan Zheng, Yang Xiang, Nicholas Jing Yuan, Xing Xie, and Zhenhui Li. 2018. DRN: A deep reinforcement learning framework for news recommendation. In Proceedings of the 2018 World Wide Web Conference. 167--176.

Cited By

View all
  • (2024)Make graph neural networks great againProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/288(2607-2615)Online publication date: 3-Aug-2024
  • (2024)Hierarchical reinforcement learning for point of interest recommendationProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/272(2460-2468)Online publication date: 3-Aug-2024
  • (2024)Mobility Prediction via Rule-enhanced Knowledge GraphACM Transactions on Knowledge Discovery from Data10.1145/367701918:9(1-21)Online publication date: 9-Oct-2024
  • Show More Cited By

Index Terms

  1. Incremental Mobile User Profiling: Reinforcement Learning with Spatial Knowledge Graph for Modeling Event Streams

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
      August 2020
      3664 pages
      ISBN:9781450379984
      DOI:10.1145/3394486
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 20 August 2020

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. incremental learning
      2. mobile user profiling
      3. reinforcement learning
      4. spatial knowledge graph

      Qualifiers

      • Research-article

      Funding Sources

      • National Science Foundation

      Conference

      KDD '20
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

      Upcoming Conference

      KDD '25

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)132
      • Downloads (Last 6 weeks)10
      Reflects downloads up to 15 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Make graph neural networks great againProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/288(2607-2615)Online publication date: 3-Aug-2024
      • (2024)Hierarchical reinforcement learning for point of interest recommendationProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/272(2460-2468)Online publication date: 3-Aug-2024
      • (2024)Mobility Prediction via Rule-enhanced Knowledge GraphACM Transactions on Knowledge Discovery from Data10.1145/367701918:9(1-21)Online publication date: 9-Oct-2024
      • (2024)Mobilytics: Mobility Analytics Framework for Transferring Semantic KnowledgeIEEE Transactions on Mobile Computing10.1109/TMC.2024.341358923:12(11588-11603)Online publication date: Dec-2024
      • (2024)Continual Learning for Smart City: A SurveyIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.344712336:12(7805-7824)Online publication date: Dec-2024
      • (2024)Vehicular Social Dynamic Anomaly Detection With Recurrent Multi-Mask Aggregator Enabled VAEIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.345756925:12(21709-21722)Online publication date: 25-Sep-2024
      • (2024)Learning Semantic Behavior for Human Mobility Trajectory RecoveryIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.335023425:8(8849-8864)Online publication date: Aug-2024
      • (2024)A Preliminary Study on Dynamic Urban Knowledge Graph Construction using Heterogeneous Spatio-temporal Data2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10825457(6834-6841)Online publication date: 15-Dec-2024
      • (2024)Temporal Inductive Path Neural Network for Temporal Knowledge Graph ReasoningArtificial Intelligence10.1016/j.artint.2024.104085(104085)Online publication date: Feb-2024
      • (2024)Promoting sustainable tourism by recommending sequences of attractions with deep reinforcement learningInformation Technology & Tourism10.1007/s40558-024-00288-x26:3(449-484)Online publication date: 24-Apr-2024
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media