skip to main content
10.1145/3488560.3498444acmconferencesArticle/Chapter ViewAbstractPublication PageswsdmConference Proceedingsconference-collections
research-article

ST-GSP: Spatial-Temporal Global Semantic Representation Learning for Urban Flow Prediction

Published: 15 February 2022 Publication History

Abstract

Urban flow prediction plays a crucial role in public transportation management and smart city construction. Although previous studies have achieved success in integrating spatial-temporal information to some extents, those models lack thoughtful consideration on global information and positional information in the temporal dimension, which can be summarized by three aspects: a) The models do not consider the relative position information of time axis, resulting in that the position features of flow maps are not effectively learned. b) They overlook the correlation among temporal dependencies of different scales, which lead to inaccurate global information representation. c) Those models only predict the flow map at the end of time sequence other than more flow maps before that, which results in neglecting parts of temporal features in the learning process. To solve the problems, we propose a novel model, Spatial-Temporal Global Semantic representation learning for urban flow Prediction (ST-GSP) in this paper. Specifically, for a), we design a semantic flow encoder that extracts relative positional information of time. Besides, the encoder captures the spatial dependencies and external factors of urban flow at each time interval. For b), we model the correlation among temporal dependencies of different scales simultaneously by using the multi-head self-attention mechanism, which can learn the global temporal dependencies. For c), inspired by the idea of self-supervised learning, we mask an urban flow map on the time sequence and predict it to pre-train a deep bidirectional learning model to catch the representation from its context. We conduct extensive experiments on two types of urban flows in Beijing and New York City to show that the proposed method outperforms state-of-the-art methods.

Supplementary Material

MP4 File (WSDM22-fp369.mp4)
Presentation video.

References

[1]
George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. 2015. Time Series Analysis: Forecasting and Control. John Wiley & Sons.
[2]
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. 2020. A Simple Framework for Contrastive Learning of Visual Representations. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event (Proceedings of Machine Learning Research, Vol. 119). PMLR, 1597-1607. http://proceedings.mlr.press/v119/chen20j.html
[3]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers). Association for Computational Linguistics, 4171-4186. https://doi.org/10.18653/v1/n19-1423
[4]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net. https://openreview.net/forum?id=YicbFdNTTy
[5]
Shengnan Guo, Youfang Lin, Shijie Li, Zhaoming Chen, and Huaiyu Wan. 2019. Deep Spatial-Temporal 3D Convolutional Neural Networks for Traffic Data Forecasting. IEEE Trans. Intell. Transp. Syst., Vol. 20, 10 (2019), 3913-3926. https://doi.org/10.1109/TITS.2019.2906365
[6]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016 . IEEE Computer Society, 770-778. https://doi.org/10.1109/CVPR.2016.90
[7]
Søren Johansen. 1991. Estimation and Hypothesis Testing of Cointegration Vectors in Gaussian Vector Autoregressive Models. Econometrica, Vol. 59, 6 (1991), 1551-1580. http://www.jstor.org/stable/2938278
[8]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings . http://arxiv.org/abs/1412.6980
[9]
Lingpeng Kong, Cyprien de Masson d'Autume, Lei Yu, Wang Ling, Zihang Dai, and Dani Yogatama. 2020. A Mutual Information Maximization Perspective of Language Representation Learning. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. https://openreview.net/forum?id=Syx79eBKwr
[10]
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. https://openreview.net/forum?id=H1eA7AEtvS
[11]
Yann LeCun, Yoshua Bengio, and Geoffrey E. Hinton. 2015. Deep learning. Nat., Vol. 521, 7553 (2015), 436-444. https://doi.org/10.1038/nature14539
[12]
Ziqian Lin, Jie Feng, Ziyang Lu, Yong Li, and Depeng Jin. 2019. DeepSTN+: Context-Aware Spatial-Temporal Neural Network for Crowd Flow Prediction in Metropolis. In The Thirty-Third AAAI Conference on Artificial Intelligence, Honolulu, Hawaii, USA, January 27 - February 1, 2019. AAAI Press, 1020-1027. https://doi.org/10.1609/aaai.v33i01.33011020
[13]
Lingbo Liu, Jiajie Zhen, Guanbin Li, Geng Zhan, Zhaocheng He, Bowen Du, and Liang Lin. 2021. Dynamic Spatial-Temporal Representation Learning for Traffic Flow Prediction. IEEE Trans. Intell. Transp. Syst. 22, 11 (2021), 7169-7183. https://doi.org/10.1109/TITS.2020.3002718
[14]
C. K. Moorthy and B. G. Ratcliffe. 1988. Short term traffic forecasting using time series methods. Transportation Planning and Technology 12, 1 (1988), 45-56. https://doi.org/10.1080/03081068808717359
[15]
Luís Moreira-Matias, João Gama, Michel Ferreira, João Mendes-Moreira, and Luís Damas. 2013. Predicting Taxi-Passenger Demand Using Streaming Data. IEEE Trans. Intell. Transp. Syst. 14, 3 (2013), 1393-1402. https://doi.org/10.1109/TITS.2013.2262376
[16]
Shashank Shekhar and Billy M. Williams. 2007. Adaptive Seasonal Time Series Models for Forecasting Short-Term Traffic Flow. Transportation Research Record 2024, 1 (2007), 116-125. https://doi.org/10.3141/2024-14
[17]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA. 5998-6008. https://proceedings.neurips.cc/paper/2017/hash/ 3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
[18]
Hong Wang, Xin Wang, Wenhan Xiong, Mo Yu, Xiaoxiao Guo, Shiyu Chang, and William Yang Wang. 2019. Self-Supervised Learning for Contextualized Extractive Summarization. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. Association for Computational Linguistics, 2221-2227. https://doi.org/10.18653/v1/p19-1214
[19]
Billy M. Williams, Priya K. Durvasula, and Donald E. Brown. 1998. Urban Freeway Traffic Flow Prediction: Application of Seasonal Autoregressive Integrated Moving Average and Exponential Smoothing Models. Transportation Research Record 1644, 1 (1998), 132-141. https://doi.org/10.3141/1644-14
[20]
Billy M. Williams and Lester A. Hoel. 2003. Modeling and Forecasting Vehicular Traffic Flow as a Seasonal ARIMA Process: Theoretical Basis and Empirical Results. Journal of Transportation Engineering 129, 6 (2003), 664-672. https: //doi.org/10.1061/(ASCE)0733-947X(2003)129:6(664)
[21]
Jiawei Wu, Xin Wang, and William Yang Wang. 2019. Self-Supervised Dialogue Learning. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. Association for Computational Linguistics, 3857-3867. https://doi.org/10.18653/v1/p19-1375
[22]
Liwei Wu, Shuqing Li, Cho-Jui Hsieh, and James Sharpnack. 2020. SSE-PT:Sequential Recommendation Via Personalized Transformer. In RecSys 2020: Fourteenth ACM Conference on Recommender Systems, Virtual Event, Brazil, September 22-26, 2020. ACM, 328-337. https://doi.org/10.1145/3383313.3412258
[23]
Ziru Xu, Yunbo Wang, Mingsheng Long, and Jianmin Wang. 2018. PredCNN:Predictive Learning with Cascade Convolutions. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden. ijcai.org, 2940-2947. https://doi.org/10.24963/ijcai.2018/408
[24]
Huaxiu Yao, Xianfeng Tang, Hua Wei, Guanjie Zheng, and Zhenhui Li. 2019. Revisiting Spatial-Temporal Similarity: A Deep Learning Framework for Traffic Prediction. In The Thirty-Third AAAI Conference on Artificial Intelligence, Honolulu, Hawaii, USA, January 27 - February 1, 2019. AAAI Press, 5668-5675. https://doi.org/10.1609/aaai.v33i01.33015668
[25]
Huaxiu Yao, Fei Wu, Jintao Ke, Xianfeng Tang, Yitian Jia, Siyu Lu, Pinghua Gong, Jieping Ye, and Zhenhui Li. 2018. Deep Multi-View Spatial-Temporal Network for Taxi Demand Prediction. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, Louisiana, USA, February 2-7, 2018. AAAI Press, 2588-2595. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16069
[26]
Junliang Yu, Hongzhi Yin, Min Gao, Xin Xia, Xiangliang Zhang, and Nguyen Quoc Viet Hung. 2021. Socially-Aware Self-Supervised Tri-Training for Recommendation. In KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 2021. ACM, 2084-2092. https://doi.org/10.1145/3447548.3467340
[27]
Junliang Yu, Hongzhi Yin, Jundong Li, Qinyong Wang, Nguyen Quoc Viet Hung, and Xiangliang Zhang. 2021. Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendation. In WWW '21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021. ACM / IW3C2, 413-424. https://doi.org/10.1145/3442381.3449844
[28]
Rose Yu, Yaguang Li, Cyrus Shahabi, Ugur Demiryurek, and Yan Liu. 2017. Deep Learning: A Generic Approach for Extreme Condition Traffic Forecasting. In Proceedings of the 2017 SIAM International Conference on Data Mining, Houston, Texas, USA, April 27-29, 2017. SIAM, 777-785. https://doi.org/10.1137/1.9781611974973.87
[29]
Junwei Zhang, Min Gao, Junliang Yu, Lei Guo, Jundong Li, and Hongzhi Yin. 2021. Double-Scale Self-Supervised Hypergraph Learning for Group Recommendation. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (Virtual Event, Queensland, Australia) (CIKM '21). Association for Computing Machinery, New York, NY, USA, 2557-2567. https://doi.org/10.1145/3459637.3482426
[30]
Junbo Zhang, Yu Zheng, and Dekang Qi. 2017. Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA. AAAI Press, 1655-1661. http://aaai.org/ocs/index.php/AAAI/ AAAI17/paper/view/14501
[31]
Junbo Zhang, Yu Zheng, Dekang Qi, Ruiyuan Li, and Xiuwen Yi. 2016. DNNbased prediction model for spatio-temporal data. In Proceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, GIS 2016, Burlingame, California, USA, October 31 - November 3, 2016. ACM, 92:1-92:4. https://doi.org/10.1145/2996913.2997016
[32]
Yu Zheng, Licia Capra, Ouri Wolfson, and Hai Yang. 2014. Urban Computing: Concepts, Methodologies, and Applications. ACM Trans. Intell. Syst. Technol. 5, 3 (2014), 38:1?38:55. https://doi.org/10.1145/2629592

Cited By

View all
  • (2025)Enhancing urban flow prediction via mutual reinforcement with multi-scale regional informationNeural Networks10.1016/j.neunet.2024.106900182:COnline publication date: 1-Feb-2025
  • (2025)PartialST: partial spatial–temporal learning for urban flow predictionNeural Computing and Applications10.1007/s00521-024-10888-7Online publication date: 8-Jan-2025
  • (2024)Tucker Decomposition-Enhanced Dynamic Graph Convolutional Networks for Crowd Flows PredictionACM Transactions on Intelligent Systems and Technology10.1145/370611616:1(1-19)Online publication date: 2-Dec-2024
  • Show More Cited By

Index Terms

  1. ST-GSP: Spatial-Temporal Global Semantic Representation Learning for Urban Flow Prediction

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining
        February 2022
        1690 pages
        ISBN:9781450391320
        DOI:10.1145/3488560
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 15 February 2022

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. spatial-temporal modeling
        2. transformer
        3. urban flow prediction

        Qualifiers

        • Research-article

        Funding Sources

        • the National Key Research and Development Program of China
        • the Overseas Returnees Innovation and Entrepreneurship Support Program of Chongqing
        • the Fundamental Research Funds for the Central Universities of Chongqing University
        • the Natural Science Foundation of Chongqing, China
        • the National Natural Science Foundation of China

        Conference

        WSDM '22

        Acceptance Rates

        Overall Acceptance Rate 498 of 2,863 submissions, 17%

        Upcoming Conference

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)150
        • Downloads (Last 6 weeks)12
        Reflects downloads up to 13 Feb 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2025)Enhancing urban flow prediction via mutual reinforcement with multi-scale regional informationNeural Networks10.1016/j.neunet.2024.106900182:COnline publication date: 1-Feb-2025
        • (2025)PartialST: partial spatial–temporal learning for urban flow predictionNeural Computing and Applications10.1007/s00521-024-10888-7Online publication date: 8-Jan-2025
        • (2024)Tucker Decomposition-Enhanced Dynamic Graph Convolutional Networks for Crowd Flows PredictionACM Transactions on Intelligent Systems and Technology10.1145/370611616:1(1-19)Online publication date: 2-Dec-2024
        • (2024)UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal PredictionProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671662(4095-4106)Online publication date: 25-Aug-2024
        • (2024)Urban Foundation Models: A SurveyProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671453(6633-6643)Online publication date: 25-Aug-2024
        • (2024)A Survey on Time-Series Pre-Trained ModelsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.347580936:12(7536-7555)Online publication date: Dec-2024
        • (2024)DroneSSL: Self-Supervised Multimodal Anomaly Detection in Internet of Drone ThingsIEEE Transactions on Consumer Electronics10.1109/TCE.2024.337644070:1(4287-4298)Online publication date: Feb-2024
        • (2024)A Novel Self-Supervised Learning-Based Anomalous Node Detection Method Based on an Autoencoder for Wireless Sensor NetworksIEEE Systems Journal10.1109/JSYST.2023.334743518:1(256-267)Online publication date: Mar-2024
        • (2024)MUSE-Net: Disentangling Multi-Periodicity for Traffic Flow Forecasting2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00103(1282-1295)Online publication date: 13-May-2024
        • (2024)STS-CCL: Spatial-Temporal Synchronous Contextual Contrastive Learning for Urban Traffic ForecastingICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10446624(6705-6709)Online publication date: 14-Apr-2024
        • Show More Cited By

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media