Abstract
Monitoring data of real-world networked systems could be sparse and irregular due to node failures or packet loss, which makes it a challenge to model the continuous dynamics of system states. Representing a network as graph, we propose a deep learning model, Adversarial Graph-Gated Differential Network (AGGDN). To accurately capture the spatial-temporal interactions and extract hidden features from data, AGGDN introduces a novel module, dynDC-ODE, which empowers Ordinary Differential Equation (ODE) with learning-based Diffusion Convolution (DC) to effectively infer relations among nodes and parameterize continuous-time system dynamics over graph. It further incorporates a Stochastic Differential Equation (SDE) module and applies it over graph to efficiently capture the underlying uncertainty of the networked systems. Different from any single differential equation model, the ODE part also works as a control signal to modulate the SDE propagation. With the recurrent running of the two modules, AGGDN can serve as an accurate online predictive model that is effective for either monitoring or analyzing the real-world networked objects. In addition, we introduce a soft masking scheme to capture the effects of partial observations caused by the random missing of data from nodes. As training a model with SDE component could be challenging, Wasserstein adversarial training is exploited to fit the complicated distribution. Extensive results demonstrate that AGGDN significantly outperforms existing methods for online prediction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, R.T.Q., Rubanova, Y., Bettencourt, J., Duvenaud, D.K.: Neural ordinary differential equations. In: Advances in Neural Information Processing Systems 31 (2018)
Choi, J., Choi, H., Hwang, J., Park, N.: Graph neural controlled differential equations for traffic forecasting (2021)
Chu, H., et al.: Neural turtle graphics for modeling city road layouts. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (October 2019)
Cui, Z., Lin, L., Pu, Z., Wang, Y.: Graph markov network for traffic forecasting with missing data. Trans. Res. Part C: Emerging Technol. 117 (2020)
De Brouwer, E., Simm, J., Arany, A., Moreau, Y.: GRU-ODE-Bayes: continuous modeling of sporadically-observed time series. In: Advances in Neural Information Processing Systems 32, pp. 7379–7390 (2019)
Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Advances in Neural Information Processing Systems 29 (2016)
Diao, Z., Wang, X., Zhang, D., Liu, Y., Xie, K., He, S.: Dynamic spatial-temporal graph convolutional neural networks for traffic forecasting. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI 2019) (Feb 2019)
Fang, Z., Long, Q., Song, G., Xie, K.: Spatial-temporal graph ODE networks for traffic flow forecasting. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & amp Data Mining. ACM (Aug 2021). https://doi.org/10.1145/3447548.3467430
Goyal, P., Chhetri, S.R., Canedo, A.: dyngraph2vec: capturing network dynamics using dynamic graph representation learning. arXiv abs/ arXiv: 1809.02657 (2018)
Goyal, P., Kamra, N., He, X., Liu, Y.: Dyngem: deep embedding method for dynamic graphs. arXiv abs/ arXiv: 1805.11273 (2018)
Hajiramezanali, E., Hasanzadeh, A., Duffield, N., Narayanan, K.R., Zhou, M., Qian, X.: Variational graph recurrent neural networks. CoRR abs/ arXiv: 1908.09710 (2019)
Huang, Z., Sun, Y., Wang, W.: Learning continuous system dynamics from irregularly-sampled partial observations. In: Advances in Neural Information Processing Systems 33 (2020)
Ioannidis, V.N., Marques, A.G., Giannakis, G.B.: A recurrent graph neural network for multi-relational data. In: 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8157–8161 (2019)
Jin, Y., JáJá, J.F.: Learning graph-level representations with gated recurrent neural networks. arXiv abs/ arXiv: 1805.07683 (2018)
Kidger, P., Morrill, J., Foster, J., Lyons, T.: Neural controlled differential equations for irregular time series (2020)
Kipf, T., Fetaya, E., Wang, K.C., Welling, M., Zemel, R.: Neural relational inference for interacting systems. In: Proceedings of the 35th International Conference on Machine Learning, pp. 2688–2697 (2018)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations (ICLR) (2017)
Kong, L., Sun, J., Zhang, C.: SDE-Net: equipping deep neural network with uncertainty estimates. In: Proceedings of the 37th International Conference on Machine Learning (2020)
Li, X., Wong, T.K.L., Chen, R.T.Q., Duvenaud, D.: Scalable gradients for stochastic differential equations. In: 23rd International Conference on Artificial Intelligence and Statistics, pp. 3870–3882 (Aug 2020)
Li, Y., Yu, R., Shahabi, C., Liu, Y.: Diffusion convolutional recurrent neural network: data-driven traffic forecasting. In: International Conference on Learning Representations (ICLR) (2018)
Li, Y., Tarlow, D., Brockschmidt, M., Zemel, R.S.: Gated graph sequence neural networks. In: International Conference on Learning Representations (ICLR) (2016)
Li, Y., Vinyals, O., Dyer, C., Pascanu, R., Battaglia, P.W.: Learning deep generative models of graphs. arXiv abs/ arXiv: 1803.03324 (2018)
Liao, R., et al.: Efficient graph generation with graph recurrent attention networks. In: Advances in Neural Information Processing Systems 32, pp. 4255–4265 (2019)
Liu, X., Xiao, T., Si, S., Cao, Q., Kumar, S., Hsieh, C.J.: Neural sde: stabilizing neural ode networks with stochastic noise (2019). https://doi.org/10.48550/ARXIV.1906.02355, https://arxiv.org/abs/1906.02355
Liu, X., Xiao, T., Si, S., Cao, Q., Kumar, S., Hsieh, C.J.: How does noise help robustness? explanation and exploration under the neural sde framework. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2020)
Liu, Y., et al.: Learning continuous-time dynamics by stochastic differential networks. ArXiv abs/ arXiv: 2006.06145 (2020)
Liu, Y., et al.: Continuous-time stochastic differential networks for irregular time series modeling. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds.) ICONIP 2021. CCIS, vol. 1516, pp. 343–351. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-92307-5_40
Manessi, F., Rozza, A., Manzo, M.: Dynamic graph convolutional networks. Pattern Recogn. 97 (2020)
Pareja, A., et al.: EvolveGCN: evolving graph convolutional networks for dynamic graphs. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI 2020) (2020)
Peluchetti, S., Favaro, S.: Infinitely deep neural networks as diffusion processes. In: 23rd International Conference on Artificial Intelligence and Statistics, vol. 108, pp. 1126–1136 (Aug 2020)
Poli, M., Massaroli, S., Park, J., Yamashita, A., Asama, H., Park, J.: Graph neural ordinary differential equations. arXiv abs/ arXiv: 1911.07532 (2019)
Poli, M., et al.: Continuous-depth neural models for dynamic graph prediction (2021). https://doi.org/10.48550/ARXIV.2106.11581, https://arxiv.org/abs/2106.11581
RTDS-Technologies-Inc.: Power hardware-in-the-loop (phil) (2022). https://www.rtds.com/applications/power-hardware-in-the-loop/
Rubanova, Y., Chen, T.Q., Duvenaud, D.K.: Latent ordinary differential equations for irregularly-sampled time series. In: Advances in Neural Information Processing Systems 32, pp. 5321–5331 (2019)
Sanchez-Gonzalez, A., Bapst, V., Cranmer, K., Battaglia, P.W.: Hamiltonian graph networks with ODE integrators. arXiv abs/ arXiv: 1909.12790 (2019)
Seo, Y., Defferrard, M., Vandergheynst, P., Bresson, X.: Structured sequence modeling with graph convolutional recurrent networks. In: The 25th International Conference on Neural Information Processing, pp. 362–373 (2018)
Shrivastava, H., et al.: Glad: learning sparse graph recovery. In: International Conference on Learning Representations (ICLR) (2020)
Simonovsky, M., Komodakis, N.: Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (July 2017)
Sun, C., Karlsson, P., Wu, J., Tenenbaum, J.B., Murphy, K.: Predicting the present and future states of multi-agent systems from partially-observed visual data. In: International Conference on Learning Representations (ICLR) (2019)
Taheri, A., Gimpel, K., Berger-Wolf, T.: Learning graph representations with recurrent neural network autoencoders. In: KDD 2018 Deep Learning Day (2018)
Tzen, B., Raginsky, M.: Neural stochastic differential equations: deep latent gaussian models in the diffusion limit. ArXiv abs/ arXiv: 1905.09883 (2019)
Tzen, B., Raginsky, M.: Theoretical guarantees for sampling and inference in generative models with latent diffusions. In: 32nd Annual Conference on Learning Theory, vol. 99, pp. 3084–3114 (Jun 2019)
Yan, T., Zhang, H., Li, Z., Xia, Y.: Stochastic graph recurrent neural network (2020). https://doi.org/10.48550/ARXIV.2009.00538, https://arxiv.org/abs/2009.00538
Ying, R., You, J., Morris, C., Ren, X., Hamilton, W.L., Leskovec, J.: Hierarchical graph representation learning with differentiable pooling. In: Advances in Neural Information Processing Systems 31, pp. 4805–4815 (2018)
You, J., Ying, R., Ren, X., Hamilton, W., Leskovec, J.: GraphRNN: generating realistic graphs with deep auto-regressive models. In: Proceedings of the 35th International Conference on Machine Learning, pp. 5708–5717 (2018)
Yu, B., Li, M., Zhang, J., Zhu, Z.: 3D graph convolutional networks with temporal graphs: a spatial information free framework for traffic forecasting arXiv: 1903.00919 (2019)
Yu, B., Yin, H., Zhu, Z.: Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In: Proceedings of he 27th International Joint Conference on Artificial Intelligence (IJCAI) (2018)
Yu, B., Yin, H., Zhu, Z.: ST-UNet: a spatio-temporal u-network for graph-structured time series modeling. arXiv abs/ arXiv: 1903.05631 (2019)
Acknowledgments
This work was supported in part by NSF under the award number ITE 2134840 and the U.S. DOE’s Office of EERE under the Solar Energy Technologies Office Award Number 38456.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Xing, Y., Wu, J., Liu, Y., Yang, X., Wang, X. (2024). AGGDN: A Continuous Stochastic Predictive Model for Monitoring Sporadic Time Series on Graphs. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14447. Springer, Singapore. https://doi.org/10.1007/978-981-99-8079-6_11
Download citation
DOI: https://doi.org/10.1007/978-981-99-8079-6_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8078-9
Online ISBN: 978-981-99-8079-6
eBook Packages: Computer ScienceComputer Science (R0)