Skip to main content
SpringerLink
Log in

Turning traffic volume imputation for persistent missing patterns with GNNs

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Traffic volume data at fixed detectors are of great importance to track the time-varying states of urban traffic, and the volume of each turning movement (i.e., continuing straight, turning left, turning right) in an intersection plays a significant role in traffic control. It is a difficult issue to determine the complete road network volume due to the limited coverage of detector deployment. Existing traffic volume imputation methods seem to be hardly tractable to address this persistent missing data problem of fixed detectors because they mainly function by utilizing spatiotemporal patterns of historical data to address the problem of intermittent missing traffic data collected from probe vehicles. Considering the unattainable temporal information of the target movement, this paper aims to develop an end-to-end graph neural network-based imputation approach to impute traffic volume data with persistent missing patterns. We regard the road network as a directed graph with each turning movement as a node and employ a meta-learning-based graph attention mechanism with graph embedding in multiple spatial granularities to fully extract the relationship between the target node and its neighbours. In addition, a multitask learning mechanism is designed to improve the accuracy of the imputation results by setting auxiliary tasks according to the volume conservation constraints. The proposed approach has been validated with extensive simulation experiments and applied to field tests for Shenzhen and Shanghai, China, with considerable accuracy and reliability, satisfying the demand for traffic control. To the best of our knowledge, this is the first time that traffic volume has been imputed for persistent missing patterns with a graph neural network-based method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://sumo.dlr.de

References

  1. Laharotte PA, Billot R, Côme E, Oukhellou L, Nantes A, Faouzi NEE (2015) Spatiotemporal analysis of bluetooth data: Application to a large urban network. IEEE Trans Intell Transp Syst 16 (3):1439–1448. https://doi.org/10.1109/TITS.2014.2367165

    Article  Google Scholar 

  2. Ke R, Li Z, Kim S, Ash J, Cui Z, Wang Y (2017) Real-time bidirectional traffic flow parameter estimation from aerial videos. IEEE Trans Intell Transp Syst 18(4):890–901. https://doi.org/10.1109/TITS.2016.2595526

    Article  Google Scholar 

  3. Liu J, Li T, Xie P, Du S, Teng F, Yang X (2020) Urban big data fusion based on deep learning: An overview. Inf. Fusion 53:123–133. https://doi.org/10.1016/j.inffus.2019.06.016

    Article  Google Scholar 

  4. Zheng Y, Capra L, Wolfson O, Yang H (2014) Urban computing: Concepts, methodologies, and applications. ACM Trans Intell Syst Technol 5(3):38:1–38:55. https://doi.org/10.1145/2629592

    Article  Google Scholar 

  5. Guo S, Lin Y, Feng N, Song C, Wan H (2019) Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019. AAAI Press, pp 922–929

  6. Yu B, Yin H, Zhu Z (2018) Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In: Lang J (ed) Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden. ijcai.org, pp 3634–3640

  7. Lin L, Li J, Chen F, Ye J, Huai J (2018) Road traffic speed prediction: A probabilistic model fusing multi-source data. IEEE Trans Knowl Data Eng 30(7):1310–1323. https://doi.org/10.1109/TKDE.2017.2718525

    Article  Google Scholar 

  8. Du S, Li T, Gong X, Horng S-J (2020) A hybrid method for traffic flow forecasting using multimodal deep learning. Int J Comput Intell Syst 13(1):85–97. https://doi.org/10.2991/ijcis.d.200120.001

    Article  Google Scholar 

  9. Qin H, Zhan X, Li Y, Yang X, Zheng Y (2021) Network-wide traffic states imputation using self-interested coalitional learning. In: KDD ’21: The 27th ACM SIGKDD conference on knowledge discovery and data mining, virtual event, Singapore, August 14-18, 2021. https://doi.org/10.1145/3447548.3467424, pp 1370–1378

  10. Tang J, Liu F, Zou Y, Zhang W, Wang Y (2017) An improved fuzzy neural network for traffic speed prediction considering periodic characteristic. IEEE Trans. Intell. Transp. Syst. 18(9):2340–2350. https://doi.org/10.1109/TITS.2016.2643005

    Article  Google Scholar 

  11. Ye J, Zhao J, Ye K, Xu C (2020) How to build a graph-based deep learning architecture in traffic domain: A survey. IEEE Trans Intell Transp Syst, 1–21, https://doi.org/10.1109/TITS.2020.3043250

  12. Chen X, He Z, Sun L (2019) A bayesian tensor decomposition approach for spatiotemporal traffic data imputation. Transportation Research Part C: Emerging Technologies 98:73–84. https://doi.org/10.1016/j.trc.2018.11.003

    Article  Google Scholar 

  13. Cui Z, Lin L, Pu Z, Wang Y (2020) Graph markov network for traffic forecasting with missing data. Transportation Research Part C: Emerging Technologies 117:102671. https://doi.org/10.1016/j.trc.2020.102671

    Article  Google Scholar 

  14. Cui Z, Ke R, Pu Z, Wang Y (2020) Stacked bidirectional and unidirectional lstm recurrent neural network for forecasting network-wide traffic state with missing values. Transportation Research Part C: Emerging Technologies 118:102674. https://doi.org/10.1016/j.trc.2020.102674

    Article  Google Scholar 

  15. Liang Y, Ke S, Zhang J, Yi X, Zheng Y (2018) Geoman: Multi-level attention networks for geo-sensory time series prediction. In: Lang J (ed) Proceedings of the Twenty-seventh international joint conference on artificial intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden. ijcai.org, pp 3428–3434

  16. Giraka O, Selvaraj V K (2019) Short-term prediction of intersection turning volume using seasonal arima model. Trans Lett Int J Trans Res 12(7):1–8

    Google Scholar 

  17. Yi X, Duan Z, Li T, Li T, Zhang J, Zheng Y (2019) Citytraffic: Modeling citywide traffic via neural memorization and generalization approach. In: Zhu W, Tao D, Cheng X, Cui P, Rundensteiner E A, Carmel D, He Q, Yu J X (eds) Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019. ACM, pp 2665–2671

  18. Liu A, Li C, Yue W, Zhou X (2018) Real-time traffic prediction: A novel imputation optimization algorithm with missing data. In: IEEE global communications conference, GLOBECOM 2018, Abu Dhabi, United Arab Emirates, December 9-13, 2018. https://doi.org/10.1109/GLOCOM.2018.8647193, pp 1–7

  19. Wang Y, Xiao Y, Xie X, Chen R, Liu H (2018) Real-time traffic pattern analysis and inference with sparse video surveillance information. In: Lang J (ed) Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden. ijcai.org, pp 3571–3577

  20. Velickovic P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. https://openreview.net/forum?id=rJXMpikCZ. OpenReview.net

  21. Zheng C, Fan X, Wang C, Qi J (2020) GMAN: A graph multi-attention network for traffic prediction. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020. https://aaai.org/ojs/index.php/AAAI/article/view/5477, pp 1234–1241

  22. Guo K, Hu Y, Qian S, Liu H, Zhang K, Sun Y, Gao J, Yin B (2021) Optimized graph convolution recurrent neural network for traffic prediction. IEEE Trans Intell Transp Syst 22 (2):1138–1149. https://doi.org/10.1109/TITS.2019.2963722

    Article  Google Scholar 

  23. Zhang Y, Cheng T, Ren Y (2019) A graph deep learning method for short-term traffic forecasting on large road networks. Comput. Aided Civ. Infrastructure Eng. 34(10):877–896. https://doi.org/10.1111/mice.12450, https://doi.org/10.1111/mice.12450

    Article  Google Scholar 

  24. Huang Y, Weng Y, Yu S, Chen X (2019) Diffusion convolutional recurrent neural network with rank influence learning for traffic forecasting. In: 18th IEEE International conference on trust, security and privacy In computing and communications / 13th IEEE international conference on big data science and engineering, TrustCom/BigDataSE 2019, Rotorua, New Zealand, August 5-8, 2019. https://doi.org/10.1109/TrustCom/BigDataSE.2019.00096, pp 678–685

  25. Kang Z, Xu H, Hu J, Pei X (2019) Learning dynamic graph embedding for traffic flow forecasting: A graph self-attentive method. In: 2019 IEEE Intelligent Transportation Systems Conference, ITSC 2019, Auckland, New Zealand, October 27-30, 2019. https://doi.org/10.1109/ITSC.2019.8917213, pp 2570–2576

  26. Ge L, Li H, Liu J, Zhou A (2019) Temporal graph convolutional networks for traffic speed prediction considering external factors. In: 20th IEEE International Conference on Mobile Data Management, MDM 2019, Hong Kong, SAR, China, June 10-13, 2019. https://doi.org/10.1109/MDM.2019.00-52, pp 234–242

  27. Zhang Z, Li M, Lin X, Wang Y, He F (2019) Multistep speed prediction on traffic networks: A deep learning approach considering spatio-temporal dependencies. Transportation Research Part C: Emerging Technologies 105:297–322. https://doi.org/10.1016/j.trc.2019.05.039, https://www.sciencedirect.com/science/article/pii/S0968090X18315389

    Article  Google Scholar 

  28. Yu JJQ, Gu J (2019) Real-time traffic speed estimation with graph convolutional generative autoencoder. IEEE Trans Intell Transp Syst 20(10):3940–3951. https://doi.org/10.1109/TITS.2019.2910560

    Article  Google Scholar 

  29. Zhang C, Yu J J Q, Liu Y (2019) Spatial-temporal graph attention networks: A deep learning approach for traffic forecasting. IEEE Access 7:166246–166256. https://doi.org/10.1109/ACCESS.2019.2953888

    Article  Google Scholar 

  30. Hu J, Guo C, Yang B, Jensen CS (2019) Stochastic weight completion for road networks using graph convolutional networks. In: 35th IEEE International Conference on Data Engineering, ICDE 2019, Macao, China, April 8-11, 2019. https://doi.org/10.1109/ICDE.2019.00116, pp 1274–1285

  31. Wang D, Zhang J, Cao W, Li J, Zheng Y (2018) When will you arrive? estimating travel time based on deep neural networks. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16657, pp 2500–2507

  32. Geng X, Li Y, Wang L, Zhang L, Yang Q, Ye J, Liu Y (2019) Spatiotemporal multi-graph convolution network for ride-hailing demand forecasting. In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019. AAAI Press, pp 3656– 3663

  33. Wang Y, Yin H, Chen H, Wo T, Xu J, Zheng K (2019) Origin-destination matrix prediction via graph convolution: a new perspective of passenger demand modeling. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019. https://doi.org/10.1145/3292500.3330877, pp 1227–1235

  34. Bai L, Yao L, Kanhere S S, Wang X, Sheng Q Z (2019) Stg2seq: Spatial-temporal graph to sequence model for multi-step passenger demand forecasting. In: Proceedings of the Twenty-eighth international joint conference on artificial intelligence, IJCAI 2019, Macao, China, August 10-16, 2019. https://doi.org/10.24963/ijcai.2019/274, pp 1981–1987

  35. Bai L, Yao L, Kanhere SS, Wang X, Liu W, Yang Z (2019) Spatio-temporal graph convolutional and recurrent networks for citywide passenger demand prediction. In: Zhu W, Tao D, Cheng X, Cui P, Rundensteiner EA, Carmel D, He Q, Yu JX (eds) Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019. ACM, pp 2293–2296

  36. Lin Y, Dai X, Li L, Wang F-Y (2019) Pattern sensitive prediction of traffic flow based on generative adversarial framework. IEEE Trans Intell Transp Syst 20(6):2395–2400. https://doi.org/10.1109/TITS.2018.2857224

    Article  Google Scholar 

  37. Hu H, Li G, Bao Z, Cui Y, Feng J (2016) Crowdsourcing-based real-time urban traffic speed estimation: From trends to speeds. In: 32nd IEEE International Conference on Data Engineering, ICDE 2016, Helsinki, Finland, May 16-20, 2016. https://doi.org/10.1109/ICDE.2016.7498298, pp 883–894

  38. Asif MT, Mitrovic N, Dauwels J, Jaillet P (2016) Matrix and tensor based methods for missing data estimation in large traffic networks. IEEE Trans Intell Transp Syst 17(7):1816–1825. https://doi.org/10.1109/TITS.2015.2507259

    Article  Google Scholar 

  39. Zhan X, Zheng Y, Yi X, Ukkusuri S V (2017) Citywide traffic volume estimation using trajectory data. IEEE Trans Knowl Data Eng 29(2):272–285. https://doi.org/10.1109/TKDE.2016.2621104

    Article  Google Scholar 

  40. Fortuin V, Baranchuk D, Rätsch G, Mandt S (2019) Deep amortized variational inference for multivariate time series imputation with latent gaussian process models. AABI 2019 Symposium

  41. Che Z, Purushotham S, Cho K, Sontag DA, Liu Y (2018) Recurrent neural networks for multivariate time series with missing values. Sci Rep 8:6085. https://doi.org/10.1038/s41598-018-24271-9

    Article  Google Scholar 

  42. Zhuang Y, Ke R, Wang Y (2019) Innovative method for traffic data imputation based on convolutional neural network. IET Intell Transp Syst 13(4):605–613. https://doi.org/10.1049/iet-its.2018.5114, https://ietresearch.onlinelibrary.wiley.com/doi/abs/10.1049/iet-its.201%8.5114

    Article  Google Scholar 

  43. Grover A, Leskovec J (2016) node2vec: Scalable feature learning for networks. In: Krishnapuram B, Shah M, Smola AJ, Aggarwal CC, Shen D, Rastogi R (eds) Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016. ACM, pp 855–864

  44. Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: online learning of social representations. In: Macskassy S A, Perlich C, Leskovec J, Wang W, Ghani R (eds) The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, New York, NY, USA - August 24 - 27, 2014. ACM, pp 701–710

  45. Le Q V, Mikolov T (2014) Distributed representations of sentences and documents. In: Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China, 21-26 June 2014. JMLR Workshop and Conference Proceedings, vol 32. JMLR.org, pp 1188–1196

  46. Pan Z, Liang Y, Wang W, Yu Y, Zheng Y, Zhang J (2019) Urban traffic prediction from spatio-temporal data using deep meta learning. In: Teredesai A, Kumar V, Li Y, Rosales R, Terzi E, Karypis G (eds) Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019. ACM, pp 1720–1730

  47. Yuan J, Zheng Y, Xie X (2012) Discovering regions of different functions in a city using human mobility and pois. In: Yang Q, Agarwal D, Pei J (eds) The 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’12, Beijing, China, August 12-16, 2012. ACM, pp 186–194

  48. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Bach F R, Blei D M (eds) Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015. JMLR Workshop and Conference Proceedings, vol 37. JMLR.org, pp 448–456

  49. Zhang J, Zheng Y, Sun J, Qi D (2020) Flow prediction in spatio-temporal networks based on multitask deep learning. IEEE Trans Knowl Data Eng 32(3):468–478. https://doi.org/10.1109/TKDE.2019.2891537

    Article  Google Scholar 

  50. Feng S, Ke J, Yang H, Ye J (2021) A multi-task matrix factorized graph neural network for co-prediction of zone-based and od-based ride-hailing demand. IEEE Trans Intell Transp Syst 1:1–13. https://doi.org/10.1109/TITS.2021.3056415

    Article  Google Scholar 

  51. Gao R, Guo X, Sun F, Dai L, Zhu J, Hu C, Li H (2019) Aggressive driving saves more time? multi-task learning for customized travel time estimation. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019. https://doi.org/10.24963/ijcai.2019/234, pp 1689–1696

  52. Li Y, Fu K, Wang Z, Shahabi C, Ye J, Liu Y (2018) Multi-task representation learning for travel time estimation. https://doi.org/10.1145/3219819.3220033, pp 1695–1704

Download references

Acknowledgements

This work is supported by Beijing Nova Program of Science and Technology (Grant No. Z191100001119031), National Natural Science Foundation of China (Grant No.U21A20468), Guangxi Key Laboratory of Cryptography and Information Security (Grant No. GCIS202111), Shanghai Pujiang Program (Grant No. 21PJD092), and Zhejiang Lab (Grant No. 2019KE0AB03 and No.2021PD0AB02).

Author information

Authors and Affiliations

  1. The State Key Laboratory of Networking and Switching, Beijing University of Posts and Telecommunications, Beijing, China

    Ruiqiang Liu, Shuai Zhao & Bo Cheng

  2. SenseTime Group Limited, Shanghai, China

    Ruiqiang Liu, Yuheng Kan, Zian Ma & Wei Wu

  3. Shanghai AI lab, Shanghai, China

    Yuheng Kan, Zian Ma & Wei Wu

Authors
  1. Ruiqiang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuheng Kan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuai Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bo Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zian Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuai Zhao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Ruiqiang Liu and Yuheng Kan contributed equally to this work.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, R., Kan, Y., Zhao, S. et al. Turning traffic volume imputation for persistent missing patterns with GNNs. Appl Intell 53, 491–508 (2023). https://doi.org/10.1007/s10489-022-03568-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03568-3

Keywords

Search

Navigation