Abstract
The probabilistic estimation for multivariate time series forecasting has recently become a trend in various research fields, such as traffic, climate, and finance. The multivariate time series can be treated as an interrelated system, and it is significant to assume each variable to be independent. However, most existing methods fail to simultaneously consider spatial dependencies and probabilistic temporal dynamics. To address this gap, we introduce the Graph Convolution Recurrent Denoising Diffusion model (GCRDD), a recurrent framework for spatial-temporal forecasting that captures both spatial dependencies and temporal dynamics. Specifically, GCRDD incorporates the structural dependency into a hidden state using the graph-modified gated recurrent unit and samples from the estimated data distribution at each time step by a graph conditional diffusion model. We reveal the comparative experiment performance of state-of-the-art models in two real-world road network traffic datasets to demonstrate it as the competitive probabilistic multivariate temporal forecasting framework.
R. Li and X. Li—These authors contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Afifi, H., Elmahdy, M., El Saban, M., Abu-Elkheir, M.: Probabilistic time series forecasting for unconventional oil and gas producing wells. In: The 2nd Novel Intelligent and Leading Emerging Sciences Conference, pp. 450–455 (2020)
an den Oord, A., Kalchbrenner, N., Espeholt, L., kavukcuoglu, K., Vinyals, O., Graves, A.: Conditional image generation with pixel CNN decoders. In: Advances in Neural Information Processing Systems. vol. 29, pp. 4790–4798 (2016)
Bai, J., et al.: A3t-gcn: attention temporal graph convolutional network for traffic forecasting. ISPRS Int. J. Geo Inf. 10(7), 485 (2021)
Chen, H., Rossi, R.A., Mahadik, K., Kim, S., Eldardiry, H.: Graph deep factors for probabilistic time-series forecasting. ACM Trans. Knowl. Discov. Data (2022)
Chung, F.R.: Spectral Graph Theory, vol. 92. American Mathematical Soc. (1997)
Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. In: Neural Information Processing Systems 2014 Workshop on Deep Learning (2014)
Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Advances in Neural Information Processing Systems, vol. 29 (2016)
Gasthaus, J., et al.: Probabilistic forecasting with spline quantile function RNNs. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1901–1910 (2019)
Graves, A.: Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013)
He, H., Zhang, Q., Bai, S., Yi, K., Niu, Z.: CATN: cross attentive tree-aware network for multivariate time series forecasting. Proc. AAAI Conf. Artif. Intell. 36(4), 4030–4038 (2022)
Hmamouche, Y., Przymus, P.M., Alouaoui, H., Casali, A., Lakhal, L.: Large multivariate time series forecasting: survey on methods and scalability. In: Utilizing Big Data Paradigms for Business Intelligence, pp. 170–197. IGI Global (2019)
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Adv. Neural. Inf. Process. Syst. 33, 6840–6851 (2020)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations (2016)
Kong, Z., Ping, W., Huang, J., Zhao, K., Catanzaro, B.: Diffwave: a versatile diffusion model for audio synthesis. In: International Conference on Learning Representations (2021)
Lai, G., Chang, W.C., Yang, Y., Liu, H.: Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st International Conference on Research & Development in Information Retrieval, pp. 95–104 (2018)
Li, Y., Yu, R., Shahabi, C., Liu, Y.: Diffusion convolutional recurrent neural network: data-driven traffic forecasting. In: International Conference on Learning Representations (2017)
Lim, B., Zohren, S.: Time-series forecasting with deep learning: a survey. Phil. Trans. R. Soc. A 379(2194), 20200209 (2021)
Patton, A.: Copula methods for forecasting multivariate time series. Handb. Econ. Forecast. 2, 899–960 (2013)
Rangapuram, S.S., Seeger, M.W., Gasthaus, J., Stella, L., Wang, Y., Januschowski, T.: Deep state space models for time series forecasting. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Rasul, K., Seward, C., Schuster, I., Vollgraf, R.: Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In: International Conference on Machine Learning, pp. 8857–8868 (2021)
Rozemberczki, B., et al.: PyTorch geometric temporal: spatiotemporal signal processing with neural machine learning models. In: Proceedings of the 30th International Conference on Information and Knowledge Management, pp. 4564–4573 (2021)
Seo, Y., Defferrard, M., Vandergheynst, P., Bresson, X.: Structured sequence modeling with graph convolutional recurrent networks. In: Cheng, L., Leung, A.C.S., Ozawa, S. (eds.) ICONIP 2018. LNCS, vol. 11301, pp. 362–373. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04167-0_33
Shang, C., Chen, J., Bi, J.: Discrete graph structure learning for forecasting multiple time series. arXiv preprint arXiv:2101.06861 (2021)
Shih, S.Y., Sun, F.K., Lee, H.Y.: Temporal pattern attention for multivariate time series forecasting. Mach. Learn. 108(8), 1421–1441 (2019)
Shuman, D.I., Narang, S.K., Frossard, P., Ortega, A., Vandergheynst, P.: The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process. Mag. 30(3), 83–98 (2013)
Song, Y., Ermon, S.: Improved techniques for training score-based generative models. Adv. Neural. Inf. Process. Syst. 33, 12438–12448 (2020)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, vol. 27 (2014)
Tashiro, Y., Song, J., Song, Y., Ermon, S.: CSDI: conditional score-based diffusion models for probabilistic time series imputation. Adv. Neural. Inf. Process. Syst. 34, 24804–24816 (2021)
Wu, Z., Pan, S., Long, G., Jiang, J., Zhang, C.: Graph wavenet for deep spatial-temporal graph modeling. In: The 28th International Joint Conference on Artificial Intelligence (IJCAI). International Joint Conferences on Artificial Intelligence Organization (2019)
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Philip, S.Y.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32(1), 4–24 (2020)
Wu, Z., Pan, S., Long, G., Jiang, J., Chang, X., Zhang, C.: Connecting the dots: multivariate time series forecasting with graph neural networks. In: The 26th ACM International Conference on Knowledge Discovery & Data Mining, pp. 753–763 (2020)
Yang, L., et al.: Diffusion models: a comprehensive survey of methods and applications. arXiv preprint arXiv:2209.00796 (2022)
Zhao, L., et al.: T-GCN: a temporal graph convolutional network for traffic prediction. IEEE Trans. Intell. Transp. Syst. 21(9), 3848–3858 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, R., Li, X., Gao, S., Choy, S.T.B., Gao, J. (2023). Graph Convolution Recurrent Denoising Diffusion Model for Multivariate Probabilistic Temporal Forecasting. In: Yang, X., et al. Advanced Data Mining and Applications. ADMA 2023. Lecture Notes in Computer Science(), vol 14176. Springer, Cham. https://doi.org/10.1007/978-3-031-46661-8_44
Download citation
DOI: https://doi.org/10.1007/978-3-031-46661-8_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-46660-1
Online ISBN: 978-3-031-46661-8
eBook Packages: Computer ScienceComputer Science (R0)