Towards a Trajectory-powered Foundation Model of Mobility
Pages 1 - 4
Abstract
This paper advocates for a geospatial foundation model based on human mobility trajectories in the built environment. Such a model would be widely applicable across many important societal domains currently addressed independently, including transportation networks, data-driven urban planning, tourism, and sustainability. Unlike existing large vision-language models, trained primarily on text and images, this foundation model should integrate the complex spatiotemporal and multimodal data inherent to mobility. This paper motivates this challenging research agenda, outlining many downstream applications that would be significantly impacted and enabled by such a model. It then explains the critical spatial, temporal, and contextual factors that such a model must capture in trajectories. Finally, it concludes with several research questions and directions, laying the foundations for future exploration in this exciting and emerging field.
References
[1]
Pasquale Balsebre, Weiming Huang, Gao Cong, and Yi Li. 2023. City Foundation Models for Learning General Purpose Representations from OpenStreetMap. arXiv e-prints (2023), arXiv-2310.
[2]
Michael Batty, Kay W Axhausen, Fosca Giannotti, Alexei Pozdnoukhov, Armando Bazzani, Monica Wachowicz, Georgios Ouzounis, and Yuval Portugali. 2012. Smart cities of the future. The European Physical Journal Special Topics 214 (2012), 481--518.
[3]
Cristian Bodnar, Wessel P Bruinsma, Ana Lucic, Megan Stanley, Johannes Brand-stetter, Patrick Garvan, Maik Riechert, Jonathan Weyn, Haiyu Dong, Anna Vaughan, et al. 2024. Aurora: A foundation model of the atmosphere. arXiv preprint arXiv:2405.13063 (2024).
[4]
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021).
[5]
Yile Chen, Xiucheng Li, Gao Cong, Zhifeng Bao, Cheng Long, Yiding Liu, Arun Kumar Chandran, and Richard Ellison. 2021. Robust road network representation learning: When traffic patterns meet traveling semantics. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 211--220.
[6]
Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. [n. d.]. A decoder-only foundation model for time-series forecasting. In Forty-first International Conference on Machine Learning.
[7]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248--255.
[8]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 NAACL-HLT. 4171--4186.
[9]
Anita Graser, Anahid Naghibzadeh-Jalali, Jasmin Lampert, Axel Weißenfeld, and Krzysztof Janowicz. 2023. Deep Learning from Trajectory Data: a Review of Deep Neural Networks and the Trajectory Data Representations to Train Them. In Big Mobility Data Analytics BMDA 2023.
[10]
Kshitij Gupta, Benjamin Thérien, Adam Ibrahim, Mats L Richter, Quentin Anthony, Eugene Belilovsky, Irina Rish, and Timothée Lesort. 2023. Continual Pre-Training of Large Language Models: How to (re) warm your model? arXiv preprint arXiv:2308.04014 (2023).
[11]
Ye Hong, Henry Martin, and Martin Raubal. 2022. How do you go where? improving next location prediction by learning travel mode information using transformers. In Proceedings of the 30th International Conference on Advances in Geographic Information Systems. 1--10.
[12]
Yan Lin, Huaiyu Wan, Shengnan Guo, and Youfang Lin. 2021. Pre-training context and time aware location embeddings from spatial-temporal trajectories for user next location prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4241--4248.
[13]
Gengchen Mai, Weiming Huang, Jin Sun, Suhang Song, Deepak Mishra, Ninghao Liu, Song Gao, Tianming Liu, Gao Cong, Yingjie Hu, et al. 2023. On the opportunities and challenges of foundation models for geospatial artificial intelligence. arXiv preprint arXiv:2304.06798 (2023).
[14]
Gengchen Mai, Krzysztof Janowicz, Yingjie Hu, Song Gao, Bo Yan, Rui Zhu, Ling Cai, and Ni Lao. 2022. A review of location encoding for GeoAI: methods and applications. International Journal of Geographical Information Science 36, 4 (2022), 639--673.
[15]
Mashaal Musleh. 2022. Towards a unified deep model for trajectory analysis. In Proceedings of the 30th International Conference on Advances in Geographic Information Systems. 1--2.
[16]
Mashaal Musleh and Mohamed F Mokbel. 2024. Let's Speak Trajectories: A Vision to Use NLP Models for Trajectory Analysis Tasks. ACM Transactions on Spatial Algorithms and Systems 10, 2 (2024), 1--25.
[17]
Chung Park, Taesan Kim, Junui Hong, Minsung Choi, and Jaegul Choo. 2023. Pre-training Contextual Location Embeddings in Personal Trajectories via Efficient Hierarchical Location Representations. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. 125--140.
[18]
Jinmeng Rao, Song Gao, Gengchen Mai, and Krzysztof Janowicz. 2023. Building Privacy-Preserving and Secure Geospatial Artificial Intelligence Foundation Models (Vision Paper). In Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems. 1--4.
[19]
Daria Reshetova, Swetava Ganguli, CV Krishnakumar Iyer, and Vipul Pandey. 2023. SeMAnD: Self-supervised anomaly detection in multimodal geospatial datasets. In Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems. 1--4.
[20]
Siqin Wang, Xiao Huang, Pengyuan Liu, Mengxi Zhang, Filip Biljecki, Tao Hu, Xiaokang Fu, Lingbo Liu, Xintao Liu, Ruomei Wang, et al. 2024. Mapping the landscape and roadmap of geospatial artificial intelligence (GeoAI) in quantitative human geography: An extensive systematic review. International Journal of Applied Earth Observation and Geoinformation 128 (2024), 103734.
[21]
Bo Yan, Krzysztof Janowicz, Gengchen Mai, and Song Gao. 2017. From itdl to place2vec: Reasoning about place type similarity and relatedness by learning embeddings from augmented spatial contexts. In Proceedings of the 25th ACM SIGSPATIAL international conference on advances in geographic information systems. 1--10.
Index Terms
- Towards a Trajectory-powered Foundation Model of Mobility
Recommendations
TrajGDM: A New Trajectory Foundation Model for Simulating Human Mobility
SIGSPATIAL '23: Proceedings of the 31st ACM International Conference on Advances in Geographic Information SystemsCapturing the universal movement pattern and simulating human mobility is one of the most important trajectory data-mining tasks. Most of the current mobility modeling methods are specially designed to solve a specific task, which leads to questions ...
A Taxonomy of Foundation Model based Systems through the Lens of Software Architecture
CAIN '24: Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AILarge language model (LLM) based chatbots, such as ChatGPT, have attracted huge interest in foundation models. It is widely believed that foundation models will serve as the fundamental building blocks for future AI systems. However, the architecture ...
Geospatial Foundation Models: Recent Advances and Applications
BigSpatial '24: Proceedings of the 12th ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial DataFoundation models are deep learning models trained on massive datasets. Recent advancements have made them capable of performing a wide range of general tasks, including language processing, summarization, code generation, problem-solving, and reasoning. ...
Comments
Information & Contributors
Information
Published In
October 2024
47 pages
ISBN:9798400711459
DOI:10.1145/3681766
- Editors:
- Jinmeng Rao,
- Emre Eftelioglu,
- Heba Aly,
- Yiqun Xie,
- Song Gao
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 18 November 2024
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Conference
SIGSPATIAL '24
Sponsor:
SIGSPATIAL '24: The 32nd ACM International Conference on Advances in Geographic Information Systems
October 29 - November 1, 2024
GA, Atlanta, USA
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 276Total Downloads
- Downloads (Last 12 months)276
- Downloads (Last 6 weeks)62
Reflects downloads up to 25 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in