skip to main content
research-article

SeqST-GAN: Seq2Seq Generative Adversarial Nets for Multi-step Urban Crowd Flow Prediction

Published: 03 June 2020 Publication History

Abstract

Citywide crowd flow data are ubiquitous nowadays, and forecasting the flow of crowds is of great importance to many real applications such as traffic management and mobility-on-demand (MOD) services. The challenges of accurately predicting urban crowd flows come from both the nonlinear spatial-temporal correlations of the crowd flow data and the complex impact of the external context factors, such as weather, holidays, and POIs. It is even more challenging for most existing one-step prediction models to make an accurate prediction across multiple future time slots. In this article, we propose a sequence-to-sequence (Seq2Seq) Generative Adversarial Nets model named SeqST-GAN to perform multi-step Spatial-Temporal crowd flow prediction of a city. Motivated by the success of GAN in video prediction, we for the first time propose an adversarial learning framework by regarding the citywide crowd flow data in successive time slots as “image frames.” Specifically, we first use a Seq2Seq model to generate a sequence of future “frame” predictions based on previous ones. Then, by integrating the generation error with the adversary loss, SeqST-GAN can avoid the blurry prediction issue and make more accurate predictions. To incorporate the external contexts, an external-context gate module called EC-Gate is also proposed to learn region-level representations of the context features. Experiments on two large crowd flow datasets in New York demonstrate that SeqST-GAN improves the prediction performance by a large margin compared with the existing state-of-the-art.

References

[1]
Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Fei-Fei, and Silvio Savarese. 2016. Social LSTM: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
[2]
Martin Arjovsky, Soumith Chintala, and Leon Bottou. 2017. Wasserstein generative adversarial networks. In Proceedings of the International Conference on Machine Learning.
[3]
Gowtham Atluri, Anuj Karpatne, and Vipin Kumar. 2017. Spatio-temporal data mining: A survey of problems and methods. arXiv:1711.04710v2. (2017).
[4]
Jimmy Ba, Volodymyr Mnih, and Koray Kavukcuoglu. 2015. Multiple object recognition with visual attention. In Proceedings of the International Conference on Learning Representations.
[5]
Dzmitry Bahdanau, KyungHyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations.
[6]
Prateep Bhattacharjee and Sukhendu Das. 2017. Temporal coherency based criteria for predicting video frames using deep multi-stage generative adversarial networks. In Proceedings of the International Conference on Advances in Neural Information Processing Sytems.
[7]
Mecit Cetin and Gurcan Comert. 2006. Short-term traffic flow prediction with regime switching models. Transport. Res. Rec.: J. Transport. Res. Board 1965 (2006).
[8]
Y Chen and D Xiao. 2009. Traffic network flow forecasting based on switching model. Contr. Decis. 24, 8 (2009), 1177--1180.
[9]
Xingyi Cheng, Ruiqing Zhang, Jie Zhou, and Wei Xu. 2017. Deep transport: Learning spatial-temporal dependency for traffic condition forecasting. In Proceedings of International Joint Conference on Neural Networks (IJCNN’18).
[10]
Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the International Conference on Machine Learning.
[11]
Emily L. Denton, Soumith Chintala, Arthur Szlam, and Rob Fergus. 2015.Deep generative image models using a Laplacian pyramid of adversarial networks. In Proceedings of the Conference on Neural Information Processing Systems.
[12]
Zipei Fan, Xuan Song, Ryosuke Shibasaki, and Ryutaro Adachi. 2015. CityMomentum: An online approach for crowd behavior prediction at a citywide level. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing.
[13]
Ian J. Goodfellow, Jean Pouget-Abadiey, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozairz, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Proceedings of the International Conference on Neural Information Processing Systems.
[14]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
[15]
Aude Hofleitner, Ryan Herring, and Pieter Abbeel. 2012. Learning the dynamics of arterial traffic from probe data using a dynamic Bayesian network. IEEE Trans. Intell. Transport. Syst. 13, 4 (2012), 1679--1693.
[16]
Alex X. Lee, Richard Zhang, Frederik Ebert, Pieter Abbeel, Chelsea Finn, and Sergey Levine. 2018. Stochastic adversarial video prediction. arXiv:1804.01523v1 (2018).
[17]
Sangsoo Lee and Daniel Fambro. 1999. Application of subset autoregressive integrated moving average model for short-term freeway traffic volume forecasting. Transport. Res. Rec.: J. Transport. Res. Board 1678 (1999).
[18]
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In Proceedings of the International Conference on Learning Representations (ICLR’18).
[19]
Xiaodan Liang, Lisa Lee, Wei Dai, and Eric P. Xing. 2017. Dual motion GAN for future-flow embedded video prediction. In Proceedings of the IEEE International Conference on Computer Vision.
[20]
M. Lippi, M. Bertini, and Paolo Frasconi. 2013. Short-term traffic flow forecasting: An experimental comparison of time-series analysis and supervised learning. IEEE Trans. Intell. Transport. Syst. 14, 2 (2013), 871--882.
[21]
Marco Lippi, Matteo Bertini, and Paolo Frasconi. 2013. Short-term traffic flow forecasting: An experimental comparison of time-series analysis and supervised learning. IEEE Trans. Intell. Transport. Syst. 14, 2 (2013), 871--882.
[22]
X. Ma, Z. Dai, Z. He, J. Ma, Y. Wang, and Y. Wang. 2017. Learning traffic as images: A deep convolutional neural network for large-scale transportation network speed prediction. Sensors 17, 4 (2017), 818.
[23]
Xudong Mao, Qing Li, Haoran Xie, Raymond Y. K. Lau, Zhen Wang, and Stephen Paul Smolley. 2017. Least squares generative adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision.
[24]
Julieta Martinez, Michael J. Black, and Javier Romero. 2017. On human motion prediction using recurrent neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
[25]
Michael Mathieu, Camille Couprie, and Yann LeCun. 2015. Deep multi-scale video prediction beyond mean square error. arXiv:1511.05440 (2015).
[26]
Takeru Miyato, Andrew M. Dai, and Ian Goodfellow. 2017. Adversarial training methods for semi-supervised text classification. In Proceedings of the International Conference on Learning Representations.
[27]
Dai Quoc Nguyen, Dat Quoc Nguyen, Cuong Xuan Chu, Stefan Thater, and Manfred Pinkal. 2017. Sequence to sequence learning for event prediciton. In Proceedings of the Annual Meeting of the Association for Computational Linguistics.
[28]
Alec Radford, Luke Metz, and Soumith Chintala. 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. In Proceedings of the International Conference on Learning Representations.
[29]
Alexandre Robicquet, Amir Sadeghian, Alexandre Alahi, and Silvio Savarese. 2016. Learning social etiquette: Human trajectory understanding in crowded scenes. In Proceedings of the European Conference on Computer Vision.
[30]
Sujit K. Sahu and Kanti V. Mardia. 2005. A Bayesian kriged Kalman model for short-term forecasting of air pollution levels. J. Roy. Statist. Soc. Series C: Appl. Statist. 54, 1 (2005), 223--244.
[31]
Shashank Shekhar and Billy Williams. 2008. Adaptive seasonal time series models for forecasting short-term traffic flow. Transport. Res. Rec.: J. Transport. Res. Board 2024, 116--125 (2008).
[32]
Xuan Song, Quanshi Zhang, Yoshihide Sekimoto, and Ryosuke Shibasaki. 2014. Prediction of human emergency behavior and their mobility following large-scale disaster. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining.
[33]
Hao Wang, Naiyan Wang, and Dit-Yan Yeung. 2015. Collaborative deep learning for recommender systems. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining.
[34]
Leye Wang, Xu Geng, Xiaojuan Ma, Feng Liu, and Qiang Yang. 2018. Crowd flow prediction by deep spatio-temporal transfer learning. arXiv:1802.00386. (2018).
[35]
Senzhang Wang, Jiannong Cao, and Philip S. Yu. 2019. Deep learning for spatio-temporal data mining: A survey. arXiv preprint arXiv:1906.04928. (2019).
[36]
Senzhang Wang, Lifang He, Leon Stenneth, Philip S. Yu, and Zhoujun Li. 2015. Citywide traffic congestion estimation with social media. In Proceedings of the ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems.
[37]
Senzhang Wang, Fengxiang Li, Leon Stenneth, and Philip S. Yu. 2016. Enhancing traffic congestion estimation with social media by coupled hidden Markov model. In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases.
[38]
Senzhang Wang, Xiaoming Zhang Fengxiang Li, Philip S. Yu, and Zhiqiu Huang. 2019. Efficient traffic estimation with multi-sourced data by parallel coupled hidden Markov model. IEEE Trans. Intell. Transport. Syst. 20, 8 (2019), 3010--3023.
[39]
Senzhang Wang, Xiaoming Zhang, Jianping Cao, Lifang He, Leon Stenneth, Philip S. Yu, Zhoujun Li, and Zhiqiu Huang. 2017. Computing urban traffic congestions by incorporating sparse GPS probe data and social media data. ACM Trans. Inf. Syst. 35, 4 (2017), 40:1--40:30.
[40]
Billy Williams. 2001. Multivariate vehicular traffic flow prediction: Evaluation of ARIMAX modeling. Transport. Res. Rec.: J. Transport. Res. Board 1776 (2001).
[41]
Huaxiu Yao, Xianfeng Tang, Hua Wei, Guanjie Zheng, Yanwei Yu, and Zhenhui Li. 2018. Modeling spatial-temporal dynamics for traffic prediction. arXiv preprint arXiv:1803.01254. (2018).
[42]
Huaxiu Yao, Fei Wu, Jintao Ke, Xianfeng Tang, Yitian Jia, Siyu Lu, Pinghua Gong, Jieping Ye, and Zhenhui Li. 2018. Deep multi-view spatial-temporal network for taxi demand prediction. In Proceedings of the AAAI Conference on Artificial Intelligence.
[43]
Shuochao Yao, Shaohan Hu, Yiran Zhao, Aston Zhang, and Tarek Abdelzaher. 2017. DeepSense: A unified deep learning framework for time-series mobile sensing data processing. In Proceedings of the World Wide Web Conference (WWW’17).
[44]
Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. 2017. SeqGAN: Sequence generative adversarial nets with policy gradient. In Proceedings of the AAAI Conference on Artificial Intelligence.
[45]
Junbo Zhang, Yu Zheng, and Dekang Qi. 2017. Deep spatio-temporal residual networks for citywide crowd flows prediction. In Proceedings of the AAAI Conference on Artificial Intelligence.
[46]
Junbo Zhang, Yu Zheng, Dekang Qi, Ruiqing Zhang, and Xiuwen Yi. 2016. DNN-based prediction model for spatio-temporal data. In Proceedings of the ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems.
[47]
Yuxuan Zhang, Senzhang Wang, Bing Chen, and Jiannong Cao. 2019. GCGAN: Generative adversarial nets with graph CNN for network-scale traffic prediction. In Proceedings of the International Joint Conference on Neural Networks. 1--8.
[48]
Xian Zhou, Yanyan Shen, Yanmin Zhu, and Linpeng Huang. 2018. Predicting multi-step citywide passenger demands using attention-based neural networks. In Proceedings of the ACM International Conference on Web Search and Data Mining.

Cited By

View all
  • (2024)Overcoming Catastrophic Forgetting in Continual Fine-Grained Urban Flow InferenceACM Transactions on Spatial Algorithms and Systems10.1145/366052310:4(1-26)Online publication date: 20-Apr-2024
  • (2024)STONE: A Spatio-temporal OOD Learning Framework Kills Both Spatial and Temporal ShiftsProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671680(2948-2959)Online publication date: 25-Aug-2024
  • (2024)One Size Fits All: A Unified Traffic Predictor for Capturing the Essential Spatial–Temporal DependencyIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.325904535:8(11317-11331)Online publication date: Aug-2024
  • Show More Cited By

Index Terms

  1. SeqST-GAN: Seq2Seq Generative Adversarial Nets for Multi-step Urban Crowd Flow Prediction

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Spatial Algorithms and Systems
      ACM Transactions on Spatial Algorithms and Systems  Volume 6, Issue 4
      December 2020
      185 pages
      ISSN:2374-0353
      EISSN:2374-0361
      DOI:10.1145/3404105
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 03 June 2020
      Online AM: 07 May 2020
      Accepted: 01 December 2019
      Revised: 01 November 2019
      Received: 01 December 2018
      Published in TSAS Volume 6, Issue 4

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Generative adversarial nets
      2. crowd flow prediction
      3. deep learning

      Qualifiers

      • Research-article
      • Research
      • Refereed

      Funding Sources

      • CCF-Tencent Open Research Fund
      • NSF of Jiangsu Province
      • Key Laboratory of Safety-Critical Software
      • Hong Kong Innovation and Technology Fund
      • Hong Kong RGC Collaborative Research Fund
      • Hong Kong Scholar Program

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)122
      • Downloads (Last 6 weeks)11
      Reflects downloads up to 20 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Overcoming Catastrophic Forgetting in Continual Fine-Grained Urban Flow InferenceACM Transactions on Spatial Algorithms and Systems10.1145/366052310:4(1-26)Online publication date: 20-Apr-2024
      • (2024)STONE: A Spatio-temporal OOD Learning Framework Kills Both Spatial and Temporal ShiftsProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671680(2948-2959)Online publication date: 25-Aug-2024
      • (2024)One Size Fits All: A Unified Traffic Predictor for Capturing the Essential Spatial–Temporal DependencyIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.325904535:8(11317-11331)Online publication date: Aug-2024
      • (2024)Joint computation offloading and resource allocation for end-edge collaboration in internet of vehicles via multi-agent reinforcement learningNeural Networks10.1016/j.neunet.2024.106621179(106621)Online publication date: Nov-2024
      • (2024)Multi-attention Generative Adversarial Network for multi-step vegetation indices forecasting using multivariate time seriesEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107563128(107563)Online publication date: Feb-2024
      • (2024)Leveraging generative AI for urban digital twins: a scoping review on the autonomous generation of urban data, scenarios, designs, and 3D city models for smart city advancementUrban Informatics10.1007/s44212-024-00060-w3:1Online publication date: 14-Oct-2024
      • (2024)Short-term origin–destination flow prediction for urban rail network: a deep learning method based on multi-source big dataComplex & Intelligent Systems10.1007/s40747-024-01391-610:4(4675-4696)Online publication date: 1-Apr-2024
      • (2024)Industrial product surface defect detection via the fast denoising diffusion implicit modelInternational Journal of Machine Learning and Cybernetics10.1007/s13042-024-02213-415:11(5091-5106)Online publication date: 11-Jul-2024
      • (2024)STA-former: encoding traffic flows with spatio-temporal associations in transformer networks for predictionCluster Computing10.1007/s10586-024-04462-y27:7(9693-9714)Online publication date: 1-Oct-2024
      • (2024)Data‐Driven Predictive Modeling of Citywide Crowd Flow for Urban Safety Management: A Case Study of Beijing, ChinaJournal of Forecasting10.1002/for.3216Online publication date: 21-Nov-2024
      • Show More Cited By

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media