Abstract
Predictive analytics on Irregularly Sampled Multivariate Time Series (IS-MTS) presents a challenging problem in many real-world applications. Previous methods have primarily focused on incorporating temporal information into prediction while little effort is made to exploit the intrinsic structural information interchange among different IS-MTS at the same or across different timestamps. Recent developments in graph-based learning have shown promise in modeling spatial and structural dependencies of graph data. However, when applied to IS-MTS, they face significant challenges due to the complex data characteristics: 1) variable time intervals between observations; 2) asynchronous time points across dimensions resulting in missing values; 3) a lack of prior knowledge of connectivity structure for information propagation. To address these challenges, we propose a multivariate temporal graph network that coherently captures structural interactions, learns time-aware dependencies, and handles challenging characteristics of IS-MTS data. Specifically, we first develop a multivariate interaction module that handles the frequent missing values and adaptively extracts graph structural relations using a novel reinforcement learning module. Second, we design a correlation-aware neighborhood aggregation mechanism to capture within and across time dependencies and structural interactions. Third, we construct a novel masked time-aware self-attention to explicitly consider timestamp information and interval irregularity for determining optimal attention weights and distinguishing the influence of observation embeddings. Based on an extensive experimental evaluation, we demonstrate that our method outperforms a variety of competitors for the IS-MTS classification task.
Z. Wang and T. Jiang—These authors contributed equally.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., Woo, W.C.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Advances in Neural Information Processing Systems, vol. 28 (2015)
Che, Z., Purushotham, S., Cho, K., Sontag, D., Liu, Y.: Recurrent neural networks for multivariate time series with missing values. Sci. Rep. 8(1), 6085 (2018)
Baytas, I.M., Xiao, C., Zhang, X., Wang, F., Jain, A.K., Zhou, J.: Patient subtyping via time-aware LSTM networks. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 65–74 (2017)
Hong, S., et al.: Holmes: health online model ensemble serving for deep learning models in intensive care units. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1614–1624 (2020)
Wang, Q., et al.: BiT-MAC: mortality prediction by bidirectional time and multi-feature attention coupled network on multivariate irregular time series. Comput. Biol. Med. 155, 106586 (2023)
Horn, M., Moor, M., Bock, C., Rieck, B., Borgwardt, K.: Set functions for time series. In: International Conference on Machine Learning, pp. 4353–4363. PMLR (2020)
Mulyadi, A.W., Jun, E., Suk, H.I.: Uncertainty-aware variational-recurrent imputation network for clinical time series. IEEE Trans. Cybern. 52(9), 9684–9694 (2021)
Wang, Y., Min, Y., Chen, X., Wu, J.: Multi-view graph contrastive representation learning for drug-drug interaction prediction. In: Proceedings of the Web Conference, pp. 2921–2933 (2021)
Li, J., et al.: Predicting path failure in time-evolving graphs. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1279–1289 (2019)
Huang, Z., Sun, Y., Wang, W.: Coupled graph ode for learning interacting system dynamics. In: The 27th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2021)
Silva, I., Moody, G., Scott, D.J., Celi, L.A., Mark, R.G.: Predicting in-hospital mortality of ICU patients: the physionet/computing in cardiology challenge 2012. In: 2012 Computing in Cardiology, pp. 245–248. IEEE (2012)
Reyna, M.A., et al.: Early prediction of sepsis from clinical data: the PhysioNet/Computing in Cardiology Challenge 2019. In: 2019 Computing in Cardiology. IEEE (2019)
Reiss, A., Stricker, D.: Introducing a new benchmarked dataset for activity monitoring. In: 2012 16th International Symposium on Wearable Computers, pp. 108–109. IEEE (2012)
Zhang, X., Zeman, M., Tsiligkaridis, T., Zitnik, M.: Graph-guided network for irregularly sampled multivariate time series. In: International Conference on Learning Representations (2022)
Clark, J.S., Bjørnstad, O.N.: Population time series: process variability, observation errors, missing values, lags, and hidden states. Ecology 85(11), 3140–3150 (2004)
Sezer, O.B., Gudelek, M.U., Ozbayoglu, A.M.: Financial time series forecasting with deep learning: a systematic literature review: 2005–2019. Appl. Soft Comput. 90, 106181 (2020)
Ma, F., Chitta, R., Zhou, J., You, Q., Sun, T. and Gao, J. Dipole: Diagnosis prediction in healthcare via attention-based bidirectional recurrent neural networks. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery And Data Mining, pp. 1903–1911 (2017)
Shickel, B., Tighe, P.J., Bihorac, A., Rashidi, P.: Deep EHR: a survey of recent advances in deep learning techniques for electronic health record (EHR) analysis. IEEE J. Biomed. Health Inform. 22(5), 1589–1604 (2017)
Cao, D., Wang, Y., Duan, J., Zhang, C., Zhu, X., Huang, C., Tong, Y., Xu, B., Bai, J., Tong, J., Zhang, Q.: Spectral temporal graph neural network for multivariate time-series forecasting. Adv. Neural. Inf. Process. Syst. 33, 17766–17778 (2020)
Wang, D., et al.: Temporal-aware graph neural network for credit risk prediction. In: Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pp. 702–710. Society for Industrial and Applied Mathematics (2021)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lió, P., Bengio, Y.: Graph attention networks. In: International Conference on Learning Representations (2018)
Chen, Z., Villar, S., Chen, L., Bruna, J.: On the equivalence between graph isomorphism testing and function approximation with GNNs. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Pei, H., Wei, B., Chang, K.C.C., Lei, Y., Yang, B.: Geom-GCN: geometric graph convolutional networks. In: International Conference on Learning Representations (2020)
Hallikainen, M., et al.: Interaction between cholesterol and glucose metabolism during dietary carbohydrate modification in subjects with the metabolic syndrome. Am. J. Clin. Nutr. 84(6), 1385–1392 (2006)
Shukla, S.N., Marlin, B.M.: A survey on principles, models and methods for learning from irregularly sampled time series. arXiv preprint arXiv:2012.00168 (2020)
Shukla, S.N., Marlin, B.: Multi-time attention networks for irregularly sampled time series. In: International Conference on Learning Representations (2020)
Tan, Q., et al.: Data-GRU: dual-attention time-aware gated recurrent unit for irregular multivariate time series. Proc. AAAI Conf. Artif. Intell. 34(01), 930–937 (2020)
Bahadori, M.T., Lipton, Z.C.: Temporal-clustering invariance in irregular healthcare time series. arXiv preprint arXiv:1904.12206 (2019)
Zhang, Y.: ATTAIN: attention-based time-aware LSTM networks for disease progression modeling. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 4369–4375, Macao, China (2019)
Yin, C., Liu, R., Zhang, D., Zhang, P.: Identifying sepsis subphenotypes via time-aware multi-modal auto-encoder. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 862–872 (2020)
Yang, S., et al.: Financial risk analysis for SMEs with graph-based supply chain mining. In: Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pp. 4661–4667 (2021)
Wang, X., et al.: Traffic flow prediction via spatial temporal graph neural network. In: Proceedings of the Web Conference, pp. 1082–1092 (2020)
Yan, C., Gao, C., Zhang, X., Chen, Y., Malin, B.: Deep imputation of temporal data. In: 2019 IEEE International Conference on Healthcare Informatics (ICHI), pp. 1–3. IEEE (2019)
Kidger, P., Morrill, J., Foster, J., Lyons, T.: Neural controlled differential equations for irregular time series. Adv. Neural. Inf. Process. Syst. 33, 6696–6707 (2020)
Rubanova, Y., Chen, R.T., Duvenaud, D.K.: Latent ordinary differential equations for irregularly-sampled time series. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Schirmer, M., Eltayeb, M., Lessmann, S., Rudolph, M.: Modeling irregular time series with continuous recurrent units. In: International Conference on Machine Learning, pp. 19388–19405. PMLR (2022)
Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8, 279–292 (1992)
Sun, Z., Sun, Z., Dong, W., Shi, J., Huang, Z.: Towards predictive analysis on disease progression: a variational Hawkes process model. IEEE J. Biomed. Health Inform. 25(11), 4195–4206 (2021)
De Brouwer, E., Simm, J., Arany, A., Moreau, Y.: GRU-ODE-Bayes: continuous modeling of sporadically-observed time series. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Oskarsson, J., Sidén, P., Lindsten, F.: Temporal graph neural networks for irregular data. In: International Conference on Artificial Intelligence and Statistics. PMLR (2023)
Acknowledgement
This research is supported by the National Key R &D Program of China (Grant No. 2022YFF0608000), the Natural Science Foundation of China (No. 62172372, No. 62272487, No. 62076178), Zhejiang Provincial Natural Science Foundation (No. LZ21F030001) and Zhejiang Lab (K2023KG0AC02).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, Z. et al. (2023). Uncovering Multivariate Structural Dependency for Analyzing Irregularly Sampled Time Series. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14173. Springer, Cham. https://doi.org/10.1007/978-3-031-43424-2_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-43424-2_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43423-5
Online ISBN: 978-3-031-43424-2
eBook Packages: Computer ScienceComputer Science (R0)
