Abstract
Accurate highway traffic forecasting is a critical task in intelligent transportation systems (ITSs), which needs to capture complex spatiotemporal dependencies from traffic sensors data. Recently, spatial-temporal graph networks have become one famous technology for various traffic forecasting tasks. Nevertheless, most of these works have assumed that the correlations between the sensors are fixed, so that it is often unable to effectively deal with the more realistic and dynamic traffic environment. To tackle this issue, we propose a joint pretraining framework for traffic flow forecasting with a gating diffusion graph attention network (JointGraph). Specifically, our proposed joint training architecture consists of two parts: network reconstructor (reconstruct a discrete graph from input data) and spatiotemporal model (forecast traffic speed with the generated network). Owing to the capabilities of the network reconstructor in generating graph structure through input node features, it is possible to apply our spatiotemporal model among multiple datasets directly. In addition, our model can expediently use information from data-rich regions and improve traffic forecasting performance on data-lacking regions in highway networks. Experiments are conducted on traffic datasets: METR-LA, PEMS-BAY, and trajectory dataset: BikeNYC. The results show that our JointGraph achieves superior performance with the state-of-the-art baselines, which further indicate that our pre-training mechanism in JointGraph provides an effective solution for multi-dataset cooperative training.






Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The datasets generated during and/or analysed during the current study are available in the https://github.com/xyk0058/ASTGAT.
References
Lee K, Eo M, Jung E, Yoon Y, Rhee W (2021) Short-term traffic prediction with deep neural networks: a survey. IEEE Access 9:54739–54756
Lv Y, Duan Y, Kang W, Li Z, Wang F-Y (2014) Traffic flow prediction with big data: a deep learning approach. IEEE Trans Intell Transp Syst 16(2):865–873
Zhang J, Zheng Y, Qi D, Li R, Yi X, Li T (2018) Predicting citywide crowd flows using deep spatio-temporal residual networks. Artif Intell 259:147–166
Wu Z, Pan S, Long G, Jiang J, Zhang C (2019) Graph wavenet for deep spatial-temporal graph modeling. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp 1907–1913
Hu J, Guo C, Yang B, Jensen CS (2019). In: Stochastic weight completion for road networks using graph convolutional networks, in: 2019 IEEE 35th International Conference on Data Engineering (ICDE0). IEEE, pp 1274–1285
Bai L, Yao L, Li C, Wang X, Wang C Adaptive graph convolutional recurrent network for traffic forecasting. Adv Neural Inf Process Syst, vol 33
Jiang D, Li G, Sun Y, Hu J, Yun J, Liu Y (2021) Manipulator grabbing position detection with information fusion of color image and depth image using deep learning. J Ambient Intell Humaniz Comput 12(12):10809–10822
Huang L, Chen C, Yun J, Sun Y, Tian J, Hao Z, Yu H, Ma H Multi-scale feature fusion convolutional neural network for indoor small target detection. Frontiers in Neurorobotics, vol 16
Jiang D, Li G, Tan C, Huang L, Sun Y, Kong J (2021) Semantic segmentation for multiscale target based on object recognition using the improved faster-rcnn model. Futur Gener Comput Syst 123:94–104
Bai D, Sun Y, Tao B, Tong X, Xu M, Jiang G, Chen B, Cao Y, Sun N, Li Z (2022) Improved single shot multibox detector target detection method based on deep feature fusion. Concurr Comput: Pract Exp 34(4):e6614
Yu R, Li Y, Shahabi C, Demiryurek U, Liu Y (2017) Deep learning: a generic approach for extreme condition traffic forecasting. In: Proceedings of the 2017 SIAM international conference on data mining. SIAM, pp 777–785
Zhang J, Zheng Y, Qi D (2017) Deep spatio-temporal residual networks for citywide crowd flows prediction. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp 1655–1661
Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Proceedings, 2005 IEEE international joint conference on neural networks, 2005. IEEE, Vol 2, pp 729–734
Bruna J, Zaremba W, Szlam A, LeCun Y (2014) Spectral networks and locally connected networks on graphs. In International Conference on Learning Representations (ICLR2014), CBLS, April 2014, pp. http-openreview
Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. Adv Neural Inf Process Syst 29:3844–3852
Max W, Kipf T N (2016) Semi-supervised classification with graph convolutional networks. In J. International Conference on Learning Representations (ICLR 2017)
Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations.
Zhang J, Shi X, Xie J, Ma H, King I, Yeung DY (2018) Gaan: gated attention networks for learning on large and spatiotemporal graphs. In: 34th conference on uncertainty in artificial intelligence. UAI 2018, 2018
Li Y, Yu R, Shahabi C, Liu Y (2018) Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In International Conference on Learning Representations
Yu B, Yin H, Zhu Z (2018) Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In: Proceedings of the 27th international joint conference on artificial intelligence, pp 3634–3640
Guo S, Lin Y, Feng N, Song C, Wan H (2019) Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In: Proceedings of the AAAI conference on artificial intelligence, Vol 33, pp 922–929
Chen W, Chen L, Xie Y, Cao W, Gao Y, Feng X (2020) Multi-range attentive bicomponent graph convolutional network for traffic forecasting. In: Proceedings of the AAAI conference on artificial intelligence, Vol 34, pp 3529–3536
Kong X, Xing W, Wei X, Bao P, Zhang J, Lu W (2020) Stgat: Spatial-temporal graph attention networks for traffic flow forecasting, vol 8, pp 134363–134372
Park C, Lee C, Bahng H, Tae Y, Jin S, Kim K, Ko S, Choo J (2020) St-grat: a novel spatio-temporal graph attention networks for accurately forecasting dynamically changing road speed. In: Proceedings of the 29th ACM international conference on information & knowledge management, pp 1215–1224
Zheng C, Fan X, Wang C, Qi J (2020) Gman: a graph multi-attention network for traffic prediction. In: Proceedings of the AAAI conference on artificial intelligence, Vol 34, pp 1234–1241
Li M, Zhu Z (2021) Spatial-temporal fusion graph neural networks for traffic flow forecasting. In: Proceedings of the AAAI conference on artificial intelligence, Vol 35, pp 4189–4196
Bai L, Yao L, Kanhere S, Yang Z, Chu J, Wang X (2019) Passenger demand forecasting with multi-task convolutional recurrent neural networks. In: Pacific-Asia conference on knowledge discovery and data mining. Springer, pp 29–42
Bai L, Yao L, Kanhere S, Wang X, Sheng Q (2019) STG2seq: Spatial-temporal graph to sequence model for multi-step passenger demand forecasting. In: Proceedings of the 28th international joint conference on artificial intelligence, pp. 1981–1987
Geng X, Li Y, Wang L, Zhang L, Yang Q, Ye J, Liu Y (2019) Spatiotemporal multi-graph convolution network for ride-hailing demand forecasting. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 3656–3663
Yu B, Li M, Zhang J, Zhu Z (2019) 3d graph convolutional networks with temporal graphs: a spatial information free framework for traffic forecasting, arXiv:1903.00919
Xing J, Kong X, Xing W, Wei X, Zhang J, Lu W (2022) Stgs: construct spatial and temporal graphs for citywide crowd flow prediction. Appl Intell, pp 1–10
Diao Z, Wang X, Zhang D, Liu Y, Xie K, He S (2019) Dynamic spatial-temporal graph convolutional neural networks for traffic forecasting. Proc AAAI Conf Artif Intell 33:890–897
Kong X, Zhang J, Wei X, Xing W, Lu W (2022) Adaptive spatial-temporal graph attention networks for traffic flow forecasting. Appl Intell 52(4):4300–4316
Lu Y, Jiang X, Fang Y, Shi C (2021) Learning to pre-train graph neural networks. In: Proceedings of the AAAI conference on artificial intelligence, Vol 35, pp 4276–4284
Grover A, Leskovec J (2016) node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 855–864
Hao Z, Wang Z, Bai D, Tao B, Tong X, Chen B Intelligent detection of steel defects based on improved split attention networks. Front Bioeng Biotechnol, vol 9
Shazeer N, Lan Z, Cheng Y, Ding N, Hou L (2020) Talking-heads attention. arXiv:2003.02436
Yin M, Yao Z, Cao Y, Li X, Zhang Z, Lin S, Hu H (2020) Disentangled non-local neural networks. In: European conference on computer vision. Springer, pp 191–207
Chen Y, Dai X, Liu M, Chen D, Yuan L, Liu Z (2020) Dynamic convolution: attention over convolution kernels. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11030–11039
Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp 7132–7141
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440
Barron JT (2019) A general and adaptive robust loss function. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4331–4339
Liebel L, Körner M (2018) Auxiliary tasks in multi-task learning, arXiv:1805.06334
Feng J, Li Y, Lin Z, Rong C, Sun F, Guo D, Jin D (2021) Context-aware spatial-temporal neural network for citywide crowd flow prediction via modeling long-range spatial dependency. ACM Trans Knowl Discovery Data (TKDD) 16(3):1–21
Kamarianakis Y, Prastacos P (2003) Forecasting traffic flow conditions in an urban network: comparison of multivariate and univariate approaches. Transp Res Rec 1857(1):74–84
Yu B, Yin H, Zhu Z (2018) Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pp. 3634–3640
Zhang Q, Chang J, Meng G, Xiang S, Pan C (2020) Spatio-temporal graph structure learning for traffic forecasting. In: Proceedings of the AAAI conference on artificial intelligence, Vol 34, pp 1177–1185
Pan Z, Liang Y, Wang W, Yu Y, Zheng Y, Zhang J (2019) Urban traffic prediction from spatio-temporal data using deep meta learning. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1720–1730
Wang X, Ma Y, Wang Y, Jin W, Wang X, Tang J, Jia C, Yu J (2020) Traffic flow prediction via spatial temporal graph neural network. In: Proceedings of the web conference 2020, pp 1082–1092
Cai L, Janowicz K, Mai G, Yan B, Zhu R (2020) Traffic transformer: capturing the continuity and periodicity of time series for traffic forecasting. Trans. GIS 24(3):736–755
Oreshkin BN, Amini A, Coyle L, Coates M (2021) Fc-gaga: fully connected gated graph architecture for spatio-temporal traffic forecasting. In: Proceedings of the AAAI conference on artificial intelligence, Vol 35, pp 9233–9241
Xingjian S, Chen Z, Wang H, Yeung D-Y, Wong W-K, Woo W-c (2015) Convolutional lstm network: a machine learning approach for precipitation nowcasting. In: Advances in neural information processing systems, pp 802–810
Feng J, Lin Z, Xia T, Sun F, Guo D, Li Y (2020) A sequential convolution network for population flow prediction with explicitly correlation modelling. In: IJCAI, pp 1331–1337
Feng J, Ziqian L, Tong X, Funing S, Diansheng G, Yong L (2021) A sequential convolution network for population flow prediction with explicitly correlation modelling. In: Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pp. 1331–1337
Bai S, Kolter JZ, Koltun V An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. Universal Language Model Fine-tuning for Text Classification
Mallick T, Balaprakash P, Rask E, Macfarlane J (2021) Transfer learning with graph neural networks for short-term highway traffic forecasting. In: 2020 25th international conference on pattern recognition (ICPR). IEEE, pp 10367–10374
Sohn K, Berthelot D, Carlini N, Zhang Z, Zhang H, Raffel CA, Cubuk ED, Kurakin A, Li C. -L. (2020) Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Adv Neural Inf Process Syst 33:596–608
Wei X, Wei X, Kong X, Lu S, Xing W, Lu W (2021) Fmixcutmatch for semi-supervised deep learning. Neural Netw 133:166–176
Acknowledgments
The authors would like to thank the National Natural Science Foundation of China (61876017, 61876018, and 61906014) for their support in this research.
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kong, X., Wei, X., Zhang, J. et al. JointGraph: joint pre-training framework for traffic forecasting with spatial-temporal gating diffusion graph attention network. Appl Intell 53, 13723–13740 (2023). https://doi.org/10.1007/s10489-022-04218-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-022-04218-4