Skip to main content
SpringerLink
Log in

Recurrent neural networks integrate multiple graph operators for spatial time series prediction

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

For multivariate time series forecasting problems, entirely using the dependencies between series is a crucial way to achieve accurate forecasting. Real-life multivariate time series often have complex time dependence, spatial dependence and high nonlinearity simultaneously, so Euclidean space is no longer sufficient to describe them. graph neural network presents a vital idea to solve this problem by modelling multivariate time series as graphs. Using the nature of graphs makes it possible to capture the dependencies between multivariate time series. However, no graph structure can perfectly characterize the relationships among multivariate time series; the facts underlying multivariate time series are much more complex. Therefore, we propose an integrated model (iGoRNN), which improves the model’s understanding of the deep relationships of multivariate time series by fusing the information captured by multiple graph operators through an integrator with a specific structure. In addition, we conducted experiments on the Metr-LA and PeMS-BAY datasets. The experimental results show that the proposed model outperforms the baseline model in three evaluation metrics, MAE, MAPE and RMSE, and can forecast complex multivariate time series.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Chen Y, Tong C, Ge Y, Lan T (2021) Fault detection based on auto-regressive extreme learning machine for nonlinear dynamic processes. Appl Soft Comput 106:107319. https://doi.org/10.1016/j.asoc.2021.107319

    Article  Google Scholar 

  2. Finesso L, Spreij P (2019) Approximation of nonnegative systems by moving averages of fixed order. Automatica 107:1–8. https://doi.org/10.1016/j.automatica.2019.05.007

    Article  MathSciNet  MATH  Google Scholar 

  3. Caliwag A, Lim W (2021) Optimal least square vector autoregressive moving average for battery state of charge estimation and forecasting. ICT Express 7(4):493–496. https://doi.org/10.1016/j.icte.2021.03.008

    Article  Google Scholar 

  4. Yao L, Ma R, Wang H (2021) Baidu index-based forecast of daily tourist arrivals through rescaled range analysis, support vector regression, and autoregressive integrated moving average. Alexandria Eng J 60(1):365–372. https://doi.org/10.1016/j.aej.2020.08.037

    Article  Google Scholar 

  5. Yang Y, Ling S (2017) Self-weighted LAD-based inference for heavy-tailed threshold autoregressive models. J Econ 197(2):368–381. https://doi.org/10.1016/j.jeconom.2016.11.009

    Article  MathSciNet  MATH  Google Scholar 

  6. Chang F, Huang H, Chan AHS, Man SS, Gong Y, Zhou H (2022) Capturing long-memory properties in road fatality rate series by an autoregressive fractionally integrated moving average model with generalized autoregressive conditional heteroscedasticity: a case study of florida, the united states, 1975–2018. J Saf Res 81:216–224. https://doi.org/10.1016/j.jsr.2022.02.013

    Article  Google Scholar 

  7. Khan NA, Ali S, Choi K (2022) An efficient and accurate multi-sensor if estimator based on doa information and order of fractional fourier transform. Entropy 24(4):452

    Article  MathSciNet  Google Scholar 

  8. Liu X, Dong X, Zhang L, Chen J, Wang C (2023) Least squares support vector regression for complex censored data. Artif Intell Med 136:102497. https://doi.org/10.1016/j.artmed.2023.102497

    Article  Google Scholar 

  9. Gao W, Xu F, Zhou Z-H (2022) Towards convergence rate analysis of random forests for classification. Artif Intell 313:103788. https://doi.org/10.1016/j.artint.2022.103788

    Article  MathSciNet  MATH  Google Scholar 

  10. Turska E, Jurga S, Piskorski J (2021) Mood disorder detection in adolescents by classification trees, random forests and xgboost in presence of missing data. Entropy 23(9):1210

    Article  Google Scholar 

  11. Lee S, Park J, Kim N, Lee T, Quagliato L (2023) Extreme gradient boosting-inspired process optimization algorithm for manufacturing engineering applications. Mater Des 226:111625. https://doi.org/10.1016/j.matdes.2023.111625

    Article  Google Scholar 

  12. Yu Y, Si X, Hu C, Zhang J (2019) A review of recurrent neural networks: lstm cells and network architectures. Neural Comput 31(7):1235–1270

    Article  MathSciNet  MATH  Google Scholar 

  13. Lombardo E, Rabe M, Xiong Y, Nierer L, Cusumano D, Placidi L, Boldrini L, Corradini S, Niyazi M, Reiner M, Belka C, Kurz C, Riboldi M, Landry G (2023) Evaluation of real-time tumor contour prediction using LSTM networks for MR-guided radiotherapy. Radiother Oncol 182:109555. https://doi.org/10.1016/j.radonc.2023.109555

    Article  Google Scholar 

  14. Mou L, Zhao P, Xie H, Chen Y (2019) T-lstm: a long short-term memory neural network enhanced by temporal information for traffic flow prediction. Ieee Access 7:98053–98060

    Article  Google Scholar 

  15. Wen T Keyes R (2019) Time series anomaly detection using convolutional neural networks and transfer learning. arXiv:1905.13628

  16. Wang S, Ren P, Takyi-Aninakwa P, Jin S, Fernandez C (2022) A critical review of improved deep convolutional neural network for multi-timescale state prediction of lithium-ion batteries. Energies 15(14):5053. https://doi.org/10.3390/en15145053

    Article  Google Scholar 

  17. Xie Y, Sun W, Ren M, Chen S, Huang Z, Pan X (2023) Stacking ensemble learning models for daily runoff prediction using 1d and 2d CNNs. Expert Syst Appl 217:119469. https://doi.org/10.1016/j.eswa.2022.119469

    Article  Google Scholar 

  18. Li R, Gao R, Suganthan PN (2023) A decomposition-based hybrid ensemble CNN framework for driver fatigue recognition. Inf Sci 624:833–848. https://doi.org/10.1016/j.ins.2022.12.088

    Article  Google Scholar 

  19. Al-Battal AF, Lerman IR, Nguyen TQ (2023) Multi-path decoder u-net: a weakly trained real-time segmentation network for object detection and localization in ultrasound scans. Comput Med Imaging Graph 102205. https://doi.org/10.1016/j.compmedimag.2023.102205

  20. Li Wang Y, Lai Y-Z, Li Q-Q, Huang S-T (2022) RU-net: an improved u-net placenta segmentation network based on ResNet. Comput Methods Prog Biomed 227:107206. https://doi.org/10.1016/j.cmpb.2022.107206

    Article  Google Scholar 

  21. Gao J, Wu J, Zhang X, Li Y, Han C, Guo C (2022) Partition and learned clustering with joined-training: active learning of GNNs on large-scale graph. Knowl-Based Syst 258:110050. https://doi.org/10.1016/j.knosys.2022.110050

    Article  Google Scholar 

  22. Shi M, Tang Y, Zhu X, Huang Y, Wilson D, Zhuang Y, Liu J (2022) Genetic-GNN: evolutionary architecture search for graph neural networks. Knowl-Based Syst 247:108752. https://doi.org/10.1016/j.knosys.2022.108752

    Article  Google Scholar 

  23. Liu K, Gao L, Khan NM, Qi L, Guan L (2021) Integrating vertex and edge features with graph convolutional networks for skeleton-based action recognition. Neurocomputing 466:190–201. https://doi.org/10.1016/j.neucom.2021.09.034

    Article  Google Scholar 

  24. Sun Y, Huang H, Yun X, Yang B, Dong K (2022) Triplet attention multiple spacetime-semantic graph convolutional network for skeleton-based action recognition. Appl Intell 52(1):113–126

    Article  Google Scholar 

  25. Liu F, Zheng J, Zheng L, Chen C (2020) Combining attention-based bidirectional gated recurrent neural network and two-dimensional convolutional neural network for document-level sentiment classification. Neurocomputing 371:39–50. https://doi.org/10.1016/j.neucom.2019.09.012

    Article  Google Scholar 

  26. Wu Z Pan S Long G Jiang J Chang X Zhang C (2020) Connecting the dots: multivariate time series forecasting with graph neural networks. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 753–763

  27. Song C, Lin Y, Guo S, Wan H (2020) Spatial-temporal synchronous graph convolutional networks: a new framework for spatial-temporal network data forecasting. Proc AAAI Conf Artif Intell 34(01):914–921. https://doi.org/10.1609/aaai.v34i01.5438

    Article  Google Scholar 

  28. Tao H, Duan Q (2023) An adaptive frame selection network with enhanced dilated convolution for video smoke recognition. Expert Syst Appl 215:119371. https://doi.org/10.1016/j.eswa.2022.119371

    Article  Google Scholar 

  29. Salehi A, Balasubramanian M (2023) DDCNet: deep dilated convolutional neural network for dense prediction. Neurocomputing 523:116–129. https://doi.org/10.1016/j.neucom.2022.12.024

    Article  Google Scholar 

  30. Liu S, Wang A, Deng X, Yang C (2022) MGNN: a multiscale grouped convolutional neural network for efficient atrial fibrillation detection. Comput Biol Med 148:105863. https://doi.org/10.1016/j.compbiomed.2022.105863

    Article  Google Scholar 

  31. Lu Y, Jiang M, Wei L, Zhang J, Wang Z, Wei B, Xia L (2021) Automated arrhythmia classification using depthwise separable convolutional neural network with focal loss. Biomed Signal Proc Control 69:102843. https://doi.org/10.1016/j.bspc.2021.102843

    Article  Google Scholar 

  32. Gan C, Wang L, Zhang Z, Wang Z (2020) Sparse attention based separable dilated convolutional neural network for targeted sentiment analysis. Knowl-Based Syst 188:104827. https://doi.org/10.1016/j.knosys.2019.06.035

    Article  Google Scholar 

  33. Bas E, Egrioglu E, Aladag CH, Yolcu U (2015) Fuzzy-time-series network used to forecast linear and nonlinear time series. Appl Intell 43(2):343–355. https://doi.org/10.1007/s10489-015-0647-0

    Article  Google Scholar 

  34. Li Y Yu R Shahabi C Liu Y (2017) Diffusion convolutional recurrent neural network: data-driven traffic forecasting. arXiv:1707.01926

  35. Zhang J Shi X Xie J Ma H King I Yeung D-Y (2018) Gaan: gated attention networks for learning on large and spatiotemporal graphs. arXiv:1803.07294

  36. Man J, Dong H, Yang X, Meng Z, Jia L, Qin Y, Xin G (2022) GCG: graph convolutional network and gated recurrent unit method for high-speed train axle temperature forecasting. Mech Syst Signal Process 163:108102. https://doi.org/10.1016/j.ymssp.2021.108102

    Article  Google Scholar 

  37. Liu Y, Song Z, Xu X, Rafique W, Zhang X, Shen J, Khosravi MR, Qi L (2022) Bidirectional gru networks-based next poi category prediction for healthcare. Int J Intell Syst 37(7):4020–4040

    Article  Google Scholar 

  38. Wu F Souza A Zhang T Fifty C Yu T Weinberger K (2019) Simplifying graph convolutional networks. In: International conference on machine learning, PMLR, pp 6861–6871

  39. Schlichtkrull M Kipf TN Bloem P Berg Rvd Titov I Welling M (2018) Modeling relational data with graph convolutional networks. In: European semantic web conference, Springer, pp 593–607

  40. Jiang Y, Lin H, Li Y, Rong Y, Cheng H, Huang X (2023) Exploiting node-feature bipartite graph in graph convolutional networks. Inf Sci 628:409–423. https://doi.org/10.1016/j.ins.2023.01.107

    Article  Google Scholar 

  41. Atwood J Towsley D (2016) Diffusion-convolutional neural networks. Adv Neural Inf Process Syst 29

  42. Mallick T, Balaprakash P, Rask E, Macfarlane J (2020) Graph-partitioning-based diffusion convolutional recurrent neural network for large-scale traffic forecasting. Transp Res Rec: J Transp Res Board 2674(9):473–488. https://doi.org/10.1177/0361198120930010

    Article  Google Scholar 

  43. Shuman DI, Narang SK, Frossard P, Ortega A, Vandergheynst P (2013) The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Proc Mag 30(3):83–98

    Article  Google Scholar 

  44. Wang Y, Wang H, Lu W, Yan Y (2023) HyGGE: hyperbolic graph attention network for reasoning over knowledge graphs. Inf Sci 630:190–205. https://doi.org/10.1016/j.ins.2023.02.050

    Article  Google Scholar 

  45. Zhang X, Zhang C, Guo J, Peng C, Niu Z, Wu X (2023) Graph attention network with dynamic representation of relations for knowledge graph completion. Expert Syst Appl 219:119616. https://doi.org/10.1016/j.eswa.2023.119616

    Article  Google Scholar 

  46. Vaswani A Shazeer N Parmar N Uszkoreit J Jones L Gomez AN Kaiser Ł Polosukhin I (2017) Attention is all you need. Advances in neural information processing systems 30

  47. Chen Y, Xiong Q, Guo Y (2022) Session-based recommendation: learning multi-dimension interests via a multi-head attention graph neural network. Appl Soft Comput 131:109744. https://doi.org/10.1016/j.asoc.2022.109744

    Article  Google Scholar 

  48. Zeng P, Hu G, Zhou X, Li S, Liu P, Liu S (2022) Muformer: a long sequence time-series forecasting model based on modified multi-head attention. Knowl-Based Syst 254:109584. https://doi.org/10.1016/j.knosys.2022.109584

    Article  Google Scholar 

  49. Williams BM, Hoel LA (2003) Modeling and forecasting vehicular traffic flow as a seasonal ARIMA process: theoretical basis and empirical results. J Transp Eng 129(6):664–672. https://doi.org/10.1061/(asce)0733-947x(2003)129:6(664)

    Article  Google Scholar 

  50. Wu C-H, Ho J-M, Lee D-T (2004) Travel-time prediction with support vector regression. IEEE Trans Intell Transp Syst 5(4):276–281

    Article  Google Scholar 

  51. Sutskever I Vinyals O Le QV (2014) Sequence to sequence learning with neural networks. Adv Neural Inf Process Syst 27

  52. Yu B Yin H Zhu Z (2017) Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. arXiv:1709.04875

  53. Guo S, Lin Y, Feng N, Song C, Wan H (2019) Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. Proceedings of the AAAI conference on artificial intelligence 33:922–929

    Article  Google Scholar 

  54. Zheng C, Fan X, Wang C, Qi J (2020) Gman: a graph multi-attention network for traffic prediction. Proceedings of the AAAI conference on artificial intelligence 34:1234–1241

    Article  Google Scholar 

Download references

Funding

This research was funded by the National Natural Science Foundation of China and General Project Fund In The Field of Equipment Development Department, grant number No.61901079, No.61403110308. The APC was funded by Dalian University.

Author information

Authors and Affiliations

  1. Communication and Network Laboratory, Dalian University, 116622, Dalian, China

    Bo Peng, Yuanming Ding, Qingyu Xia & Yang Yang

  2. School of Information Engineering, Dalian University, 116622, Dalian, China

    Bo Peng, Yuanming Ding, Qingyu Xia & Yang Yang

Authors
  1. Bo Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuanming Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qingyu Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuanming Ding.

Ethics declarations

Conflicts of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, B., Ding, Y., Xia, Q. et al. Recurrent neural networks integrate multiple graph operators for spatial time series prediction. Appl Intell 53, 26067–26078 (2023). https://doi.org/10.1007/s10489-023-04632-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04632-2

Keywords

Search

Navigation