Skip to main content
SpringerLink
Log in

Adaptive spatial-temporal graph attention networks for traffic flow forecasting

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Traffic flow forecasting, which requires modelling involuted spatial and temporal dependence and uncertainty regarding road networks and traffic conditions, is a challenge for intelligent transportation systems (ITS). Recent studies have mainly focused on modelling spatial-temporal dependence through a fixed weighted graph based on prior knowledge. However, collecting up-to-date and accurate road information is costly. Moreover, is a single fixed graph enough to describe the correlation between sensors? The fixed weighted graph cannot directly relate to prediction tasks, which may result in considerable biases. To tackle this issue, in this paper, we propose a novel deep learning model framework: an adaptive spatial-temporal graph attention network (ASTGAT). Our ASTGAT simultaneously learns the dynamic graph structure and spatial-temporal dependency for traffic flow forecasting. Specifically, our framework consists of two joint training parts: a Network Generator model that generates a discrete graph with the Gumbel-Softmax technique and a Spatial-Temporal model that utilizes the generated network to predict traffic speed. Our Network Generator can adaptively infer the hidden correlations from data. Moreover, we propose a graph talking-heads attention layer (GTHA) for capturing spatial dependencies and design a gate temporal convolution (GTCN) layer for handling long temporal sequences. We evaluated our ASTGAT on two public datasets: METR-LA is collected in Los Angeles and PEMS-BAY is collected in California. Experimental results indicate that our ASTGAT outperforms the state-of-the-art (SOTA) baselines. Finally, to further describe our model, we visualize the forecasting results and the generated graph.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Lv Y, Duan Y, Kang W, Li Z, Wang FY (2014) Traffic flow prediction with big data: a deep learning approach. IEEE Trans Intell Transp Syst 16(2):865

    Google Scholar 

  2. Wang Y, Zhang D, Liu Y, Dai B, Lee LH (2019) Enhancing transportation systems via deep learning: A survey. Transp Res C Emerg Technol 99:144. https://doi.org/10.1016/j.trc.2018.12.004

    Article  Google Scholar 

  3. Yu H, Wu Z, Wang S, Wang Y, Ma X (2017) Spatiotemporal recurrent convolutional networks for traffic prediction in transportation networks. Sensors 17(7):1501

    Article  Google Scholar 

  4. Tan H, Wu Y, Shen B, Jin PJ, Ran B (2016) Short-term traffic prediction based on dynamic tensor completion. IEEE Trans Intell Transp Syst 17(8):2123

    Article  Google Scholar 

  5. Ma X, Dai Z, He Z, Ma J, Wang Y, Wang Y (2017) Learning traffic as images: a deep convolutional neural network for large-scale transportation network speed prediction. Sensors 17(4):818

    Article  Google Scholar 

  6. Zhang J, Zheng Y, Qi D, Li R, Yi X, Li T (2018) Predicting citywide crowd flows using deep spatio-temporal residual networks. Artif Intell 259:147

    Article  MathSciNet  Google Scholar 

  7. Kipf TN, Welling M (2017) Semi-Supervised classification with graph convolutional networks. In: International conference on learning representations (ICLR)

  8. Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations. https://openreview.net/forum?id=rJXMpikCZ

  9. Seo Y, Defferrard M, Vandergheynst P, Bresson X (2018) Structured sequence modeling with graph convolutional recurrent networks. In: Cheng L, Leung ACS, Ozawa S (eds) Neural information processing. Springer International Publishing, Cham, pp 362–373

  10. Li Y, Yu R, Shahabi C, Liu Y (2018) Diffusion convolutional recurrent neural network: data-driven traffic forecasting. https://openreview.net/forum?id=SJiHXGWAZ

  11. Pan Z, Liang Y, Wang W, Yu Y, Zheng Y, Zhang J (2019) Urban traffic prediction from spatio-temporal data using deep meta learning. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1720–1730

  12. Yu B, Yin H, Zhu Z (2018) Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. pp 3634–3640. http://dl.acm.org/citation.cfm?id=3304222.3304273

  13. Wu Z, Pan S, Long G, Jiang J, Zhang C (2019) Graph WaveNet for deep spatial-temporal graph modeling. In: The 28th international joint conference on artificial intelligence (IJCAI) (International Joint Conferences on Artificial Intelligence Organization

  14. Haklay M, Weber P (2008) Openstreetmap: User-generated street maps. IEEE Pervasive Comput 7(4):12

    Article  Google Scholar 

  15. Yu B, Li M, Zhang J, Zhu Z (2019) 3d graph convolutional networks with temporal graphs: A spatial information free framework for traffic forecasting. arXiv:1903.00919

  16. Li M, Zhu Z (2020) Spatial-temporal fusion graph neural networks for traffic flow forecasting. arXiv:2012.09641

  17. Geng X, Li Y, Wang L, Zhang L, Yang Q, Ye J, Liu Y (2019) Spatiotemporal multi-graph convolution network for ride-hailing demand forecasting. In: Proceedings of the AAAI conference on artificial intelligence, vol. 33. pp 3656–3663

  18. Chen Y, Wu L, Zaki M (2020) Iterative deep graph learning for graph neural networks: better and robust node embeddings. Adv Neural Inf Process Syst 33

  19. Franceschi L, Niepert M, Pontil M, He X (2019) Learning discrete structures for graph neural networks. In: Chaudhuri K, Salakhutdinov R (eds) Proceedings of the 36th international conference on machine learning, proceedings of machine learning research, (PMLR). http://proceedings.mlr.press/v97/franceschi19a.html, vol 97, pp 1972–1982

  20. Zhang Z, Zhao Y, Liu J, Wang S, Tao R, Xin R, Zhang J (2019) A general deep learning framework for network reconstruction and dynamics learning. Appl Netw Sci 4(1):1

    Article  Google Scholar 

  21. Jang E, Gu S, Poole B. (2016) Categorical reparameterization with gumbel-softmax. arXiv:1611.01144

  22. Shazeer N, Lan Z, Cheng Y, Ding N, Hou L (2020) Talking-heads attention. arXiv:2003.02436

  23. Abbasi M, Shahraki A, Taherkordi A (2021) Deep learning for network traffic monitoring and analysis (ntma): a survey. Comput Commun

  24. Lin K, Xu X, Gao H (2021) TSCRNN: A novel classification scheme of encrypted traffic based on flow spatiotemporal features for efficient management of IIoT. Comput Netw 190:107974

    Article  Google Scholar 

  25. Gao H, Liu C, Li Y, Yang X (2020) V2VR: reliable hybrid-network-oriented V2V data transmission and routing considering RSUs and connectivity probability. IEEE Trans Intell Transp Syst

  26. Nishi T, Otaki K, Hayakawa K, Yoshimura T (2018) Traffic signal control based on reinforcement learning with graph convolutional neural nets. In: 2018 21st International conference on intelligent transportation systems (ITSC). IEEE, pp 877–883

  27. Zhang Y, Zhou Y, Lu H, Fujita H (2021) Spark cloud-based parallel computing for traffic network flow predictive control using non-analytical predictive model. IEEE Trans Intell Transp Syst

  28. Bi Z, Yu L, Gao H, Zhou P, Yao H (2020) Improved VGG model-based efficient traffic sign recognition for safe driving in 5G scenarios. Int J Mach Learn Cybern :1–12

  29. Kuang L, Hua C, Wu J, Yin Y, Gao H (2020) Traffic volume prediction based on multi-sources GPS trajectory data by temporal convolutional network. Mobile Netw Appl 25(4):1405

    Article  Google Scholar 

  30. Nagy AM, Simon V (2018) Survey on traffic prediction in smart cities. Pervasive Mob Comput : S1574119217306,521–

  31. Ahmed MS, Cook AR (1979) Analysis of freeway traffic time-series data by using Box-Jenkins techniques. 722

  32. Kamarianakis Y, Prastacos P (2003) Forecasting traffic flow conditions in an urban network: Comparison of multivariate and univariate approaches. Transp Res Rec 1857(1):74

    Article  Google Scholar 

  33. Van Der Voort M, Dougherty M, Watson S (1996) Combining Kohonen maps with ARIMA time series models to forecast traffic flow. Transp Res C Emerg Technol 4(5):307

    Article  Google Scholar 

  34. Min W, Wynter L (2011) Real-time road traffic prediction with spatio-temporal correlations. Transp Res C Emerg Technol 19(4):606

    Article  Google Scholar 

  35. Chien SIJ, Kuchipudi CM (2003) Dynamic travel time prediction with real-time and historic data. J Transp Eng 129(6):608

    Article  Google Scholar 

  36. Anand A, Ramadurai G, Vanajakshi L (2014) Data fusion-based traffic density estimation and prediction. J Intell Transp Syst 18(4):367

    Article  Google Scholar 

  37. Nikovski D, Nishiuma N, Goto Y, Kumazawa H (2005) Univariate short-term prediction of road travel times. In: Proceedings. 2005 IEEE Intelligent transportation systems, 2005. IEEE, pp 1074–1079

  38. Li S, Shen Z, Wang FY (2012) A weighted pattern recognition algorithm for short-term traffic flow forecasting. In: Proceedings of 2012 9th IEEE international conference on networking, sensing and control. IEEE, pp 1–6

  39. Ma X, Tao Z, Wang Y, Yu H, Wang Y (2015) Long short-term memory neural network for traffic speed prediction using remote microwave sensor data. Transp Res C Emerg Technol 54:187

    Article  Google Scholar 

  40. Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555

  41. Zhang J, Shi X, Xie J, Ma H, King I, Yeung D (2018) GaAN: Gated attention networks for learning on large and spatiotemporal graphs. arXiv:1803.07294

  42. Guo S, Lin Y, Feng N, Song C, Wan H (2019) Multi-range attentive bicomponent graph convolutional network for traffic forecasting. In: Proceedings of the AAAI conference on artificial intelligence. https://doi.org/10.1609/aaai.v33i01.3301922, vol 33, p 922

  43. Chen W, Chen L, Xie Y, Cao W, Gao Y, Feng X (2020) Multi-range attentive bicomponent graph convolutional network for traffic forecasting. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 3529–3536

  44. Zheng C, Fan X, Wang C, Qi J (2020) GMAN: a graph multi-attention network for traffic prediction. In: AAAI, pp 1234– 1241

  45. Park C, Lee C, Bahng H, Tae Y, Jin S, Kim K, Ko S, Choo J (2020) ST-GRAT: a novel spatio-temporal graph attention networks for accurately forecasting dynamically changing road speed. In: Proceedings of the 29th ACM international conference on information & knowledge management, pp 1215–1224

  46. Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: IEEE International joint conference on neural networks

  47. Franco S, Marco G, Chung TAh, Markus H, Gabriele M (2009) The graph neural network model. IEEE Trans Neural Netw 20(1):61

    Article  Google Scholar 

  48. Wu Z, Pan S, Chen F, Long G, Zhang C, Philip SY (2020) A comprehensive survey on graph neural networks. IEEE Trans Neural Netw Learn Syst

  49. Bruna J, Zaremba W, Szlam A, LeCun Y (2013) Spectral networks and locally connected networks on graphs

  50. Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. In: Lee DD, Sugiyama M, Luxburg UV, Guyon I, Garnett R (eds) Advances in neural information processing systems, vol 29. Curran associates Inc., pp 3844–3852

  51. Bahdanau D, Cho KH, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: 3rd international conference on learning representations, ICLR 2015

  52. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł., Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  53. Iida S, Kimura R, Cui H, Hung PH, Utsuro T, Nagata M (2019) Attention over heads: A multi-hop attention for neural machine translation. In: Proceedings of the 57th annual meeting of the association for computational linguistics: student research workshop, pp 217–222

  54. Shuman DI, Narang SK, Frossard P, Ortega A, Vandergheynst P (2013) The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process Mag 30(3):83

    Article  Google Scholar 

  55. Grover A, Leskovec J (2016) node2vec: Scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pp 855–864

  56. Nadarajah S, Kotz S (2004) The beta Gumbel distribution. Math Probl Eng 2004(4):323

    Article  MathSciNet  Google Scholar 

  57. Kong X, Xing W, Wei X, Bao P, Zhang J, Lu W (2020) STGAT: spatial-temporal graph attention networks for traffic flow forecasting. IEEE Access 8:134363

    Article  Google Scholar 

  58. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  59. Jagadish HV, Gehrke J, Labrinidis A, Papakonstantinou Y, Patel JM, Ramakrishnan R, Shahabi C (2014) Big data and its technical challenges. Commun Acm 57(7):86

    Article  Google Scholar 

  60. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. Adv Neural Inf Process Syst :3104–3112

  61. Oreshkin BN, Amini A, Coyle L, Coates MJ (2021) FC-GAGA: Fully connected gated graph architecture for spatio-temporal traffic forecasting. In: AAAI

  62. Kingma DP, Ba J (2015) Adam: A Method for Stochastic Optimization. In: ICLR (Poster)

Download references

Acknowledgements

The authors would like to thank the National Natural Science Foundation of China (61876017, 61876018, 61906014) for their support in this research.

Author information

Authors and Affiliations

  1. School of Software Engineering, Beijing Jiaotong University, Beijing, China

    Xiangyuan Kong, Jian Zhang, Xiang Wei, Weiwei Xing & Wei Lu

Authors
  1. Xiangyuan Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiang Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weiwei Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Lu.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kong, X., Zhang, J., Wei, X. et al. Adaptive spatial-temporal graph attention networks for traffic flow forecasting. Appl Intell 52, 4300–4316 (2022). https://doi.org/10.1007/s10489-021-02648-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02648-0

Keywords

Search

Navigation