skip to main content
10.1145/3397536.3422244acmconferencesArticle/Chapter ViewAbstractPublication PagesgisConference Proceedingsconference-collections
research-article

Intercity Simulation of Human Mobility at Rare Events via Reinforcement Learning

Published: 13 November 2020 Publication History

Abstract

Agent-based simulations, combined with large scale mobility data, have been an effective method for understanding urban scale human dynamics. However, collecting such large scale human mobility datasets are especially difficult during rare events (e.g., natural disasters), reducing the performance of agent-based simulations. To tackle this problem, we develop an agent-based model that can simulate urban dynamics during rare events by learning from other cities using inverse reinforcement learning. More specifically, in our framework, agents imitate real human-beings' travel behavior from areas where rare events have occurred in the past (source area) and produce synthetic people movement in different cities where such rare events have never occurred (target area). Our framework contains three main stages: 1) recovering the reward function, where the people's travel patterns and preferences are learned from the source areas; 2) transferring the model of the source area to the target areas; 3) simulating the people movement based on learned model in the target area. We apply our approach in various cities for both normal and rare situations using real-world GPS data collected from more than 1 million people in Japan, and show higher simulation performance than previous models.

References

[1]
Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, and Jorge L Reyes-Ortiz. 2012. Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In International workshop on ambient assisted living. Springer, 216--223.
[2]
Zipei Fan, Xuan Song, Ryosuke Shibasaki, Tao Li, and Hodaka Kaneda. 2016. CityCoupling: bridging intercity human mobility. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 718--728.
[3]
Jie Feng, Yong Li, Chao Zhang, Funing Sun, Fanchao Meng, Ang Guo, and Depeng Jin. 2018. Deepmove: Predicting human mobility with attentional recurrent networks. In Proceedings of the 2018 world wide web conference. 1459--1468.
[4]
Marta C Gonzalez, Cesar A Hidalgo, and Albert-Laszlo Barabasi. 2008. Understanding individual human mobility patterns. nature 453, 7196 (2008), 779--782.
[5]
Qi Han, Theo Arentze, Harry Timmermans, Davy Janssens, and Geert Wets. 2008. Modelling Context-Sensitive Dynamic Activity-Travel Behavior Under Conditions of Uncertainty Incorporating Reinforcement Learning, Habit Formation, And Behavioral and Cognitive Adaptation Strategies. Transportation Research Board.
[6]
Tianfu He, Jie Bao, Ruiyuan Li, Sijie Ruan, Yanhua Li, Li Song, Hui He, and Yu Zheng. 2020. What is the Human Mobility in a New City: Transfer Mobility Knowledge Across Cities. In Proceedings of The Web Conference 2020 (WWW '20). Association for Computing Machinery, New York, NY, USA, 1355--1365. https://doi.org/10.1145/3366423.3380210
[7]
Renhe Jiang, Xuan Song, Zipei Fan, Tianqi Xia, Quanjun Chen, Satoshi Miyazawa, and Ryosuke Shibasaki. 2018. DeepUrbanMomentum: An Online Deep-Learning System for Short-Term Urban Mobility Prediction.
[8]
Takehiro Kashiyama, Yanbo Pang, and Yoshihide Sekimoto. 2017. Open PFLOW: Creation and evaluation of an open dataset for typical people mass movement in urban areas. Transportation research part C: emerging technologies 85 (2017), 249--267.
[9]
Jens Kober, J Andrew Bagnell, and Jan Peters. 2013. Reinforcement learning in robotics: A survey. The International Journal of Robotics Research 32, 11 (2013), 1238--1274.
[10]
Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. 2013. Domain generalization via invariant feature representation. In International Conference on Machine Learning. 10--18.
[11]
Andrew Y Ng, Stuart J Russell, et al. 2000. Algorithms for inverse reinforcement learning. In Icml, Vol. 1. 2.
[12]
Yanbo Pang, Kota Tsubouchi, Takahiro Yabe, and Yoshihide Sekimoto. 2018. Replicating urban dynamics by generating human-like agents from smartphone GPS data. In Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 440--443.
[13]
Pratap S Prasad and Prathima Agrawal. 2010. Movement prediction in wireless networks using mobility traces. In 2010 7th IEEE Consumer Communications and Networking Conference. IEEE, 1--5.
[14]
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017).
[15]
Chaoming Song, Zehui Qu, Nicholas Blumm, and Albert-László Barabási. 2010. Limits of predictability in human mobility. Science 327, 5968 (2010), 1018--1021.
[16]
Xuan Song, Hiroshi Kanasugi, and Ryosuke Shibasaki. 2016. DeepTransport: Prediction and Simulation of Human Mobility and Transportation Mode at a Citywide Level. In IJCAI, Vol. 16. 2618--2624.
[17]
Xuan Song, Quanshi Zhang, Yoshihide Sekimoto, and Ryosuke Shibasaki. 2014. Prediction of human emergency behavior and their mobility following large-scale disaster. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 5--14.
[18]
Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.
[19]
Jameson L Toole, Serdar Colak, Bradley Sturt, Lauren P Alexander, Alexandre Evsukoff, and Marta C González. 2015. The path most traveled: Travel demand estimation using big data resources. Transportation Research Part C: Emerging Technologies 58 (2015), 162--177.
[20]
Leye Wang, Xu Geng, Xiaojuan Ma, Feng Liu, and Qiang Yang. 2019. Cross-city Transfer Learning for Deep Spatio-temporal Prediction. In IJCAI International Joint Conference on Artificial Intelligence. 1893.
[21]
Ying Wei, Yu Zheng, and Qiang Yang. 2016. Transfer knowledge between cities. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1905-1914.
[22]
Apichon Witayangkurn, Teerayut Horanont, Natsumi Ono, Yoshihide Sekimoto, and Ryosuke Shibasaki. [n.d.]. Trip reconstruction and transportation mode extraction on low data rate GPS data from mobile phone.
[23]
Xin Wu, Jifu Guo, Kai Xian, and Xuesong Zhou. 2018. Hierarchical travel demand estimation using multiple data sources: A forward and backward propagation algorithmic framework on a layered computational graph. Transportation Research Part C: Emerging Technologies 96 (2018), 321--346.
[24]
Takahiro Yabe, Kota Tsubouchi, and Yoshihide Sekimoto. 2017. CityFlowFragility: Measuring the Fragility of People Flow in Cities to Disasters using GPS Data Collected from Smartphones. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 1--17.
[25]
Takahiro Yabe, Kota Tsubouchi, Toru Shimizu, Yoshihide Sekimoto, and Satish V Ukkusuri. 2019. City2City: Translating Place Representations across Cities. In Proceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 412--415.
[26]
Takahiro Yabe, Kota Tsubouchi, Akihito Sudo, and Yoshihide Sekimoto. 2016. Predicting irregular individual movement following frequent mid-level disasters using location data from smartphones. In Proceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 1--4.
[27]
Pang Yanbo, Yabe Takahiro, Tsubouchi Kota, Kashiyama Takehiro, and Yoshihide Sekimoto. 2020. Development of a people mass movement simulation framework based on reinforcement learning. Transportation research part C: emerging technologies (2020).
[28]
Xiong Yiliang. 2013. Modelling individual and household activity-travel scheduling behaviours in stochastic transportation networks. (2013).
[29]
Mogeng Yin, Madeleine Sheehan, Sidney Feygin, Jean-François Paiement, and Alexei Pozdnoukhov. 2017. A generative model of urban activities from cellular data. IEEE Transactions on Intelligent Transportation Systems 19, 6 (2017), 1682--1696.
[30]
Yuyu Zhang, Hanjun Dai, Chang Xu, Jun Feng, Taifeng Wang, Jiang Bian, Bin Wang, and Tie-Yan Liu. 2014. Sequential click prediction for sponsored search with recurrent neural networks. In Twenty-Eighth AAAI Conference on Artificial Intelligence.
[31]
Yu Zheng, Like Liu, Longhao Wang, and Xing Xie. 2008. Learning transportation mode from raw gps data for geographic applications on the web. In Proceedings of the 17th international conference on World Wide Web. 247--256.
[32]
Yu Zheng, Lizhu Zhang, Xing Xie, and Wei-Ying Ma. 2009. Mining interesting locations and travel sequences from GPS trajectories. In Proceedings of the 18th international conference on World wide web. 791--800.
[33]
Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, and Anind K Dey. 2008. Maximum entropy inverse reinforcement learning. In Aaai, Vol. 8. Chicago, IL, USA, 1433--1438.

Cited By

View all
  • (2025)Mobile phone data for anticipating displacements: practices, opportunities, and challengesData & Policy10.1017/dap.2024.947Online publication date: 8-Jan-2025
  • (2024)Generating Trajectories from Implicit Neural Models2024 25th IEEE International Conference on Mobile Data Management (MDM)10.1109/MDM61037.2024.00036(129-138)Online publication date: 24-Jun-2024
  • (2024)Feasibility of anomalous event detection based on Mobile Spatial Statistics: A study of six cases in JapanInternational Journal of Disaster Risk Reduction10.1016/j.ijdrr.2024.104625110(104625)Online publication date: Aug-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGSPATIAL '20: Proceedings of the 28th International Conference on Advances in Geographic Information Systems
November 2020
687 pages
ISBN:9781450380195
DOI:10.1145/3397536
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 November 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Human Mobility
  2. People Flow Simulation
  3. Reinforcement Learning
  4. Urban Computing

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

SIGSPATIAL '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 257 of 1,238 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)87
  • Downloads (Last 6 weeks)15
Reflects downloads up to 08 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Mobile phone data for anticipating displacements: practices, opportunities, and challengesData & Policy10.1017/dap.2024.947Online publication date: 8-Jan-2025
  • (2024)Generating Trajectories from Implicit Neural Models2024 25th IEEE International Conference on Mobile Data Management (MDM)10.1109/MDM61037.2024.00036(129-138)Online publication date: 24-Jun-2024
  • (2024)Feasibility of anomalous event detection based on Mobile Spatial Statistics: A study of six cases in JapanInternational Journal of Disaster Risk Reduction10.1016/j.ijdrr.2024.104625110(104625)Online publication date: Aug-2024
  • (2023)GODDAG: Generating Origin-Destination Flow for New Cities Via Domain Adversarial TrainingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.326840935:10(10048-10057)Online publication date: 1-Oct-2023
  • (2022)GeoSpread: an Epidemic Spread Modeling Tool for COVID-19 Using Mobility DataProceedings of the 2022 ACM Conference on Information Technology for Social Good10.1145/3524458.3547257(125-131)Online publication date: 7-Sep-2022
  • (2022)Exploring Human Mobility Patterns and Travel Behavior: A Focus on Private CarsIEEE Intelligent Transportation Systems Magazine10.1109/MITS.2021.309862714:5(129-146)Online publication date: Sep-2022
  • (2022)Spatial Attention Based Grid Representation Learning For Predicting Origin–Destination Flow2022 IEEE International Conference on Big Data (Big Data)10.1109/BigData55660.2022.10021023(485-494)Online publication date: 17-Dec-2022
  • (2021)Simulating Human Mobility with Agent-based Modeling and Particle Filter Following Mobile Spatial StatisticsProceedings of the 29th International Conference on Advances in Geographic Information Systems10.1145/3474717.3484203(411-414)Online publication date: 2-Nov-2021
  • (2021)Machine Learning Meets Big Spatial Data (Revised)2021 22nd IEEE International Conference on Mobile Data Management (MDM)10.1109/MDM52706.2021.00014(5-8)Online publication date: Jun-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media