Abstract
The development of Internet of Things (IoT) technology has led to increasingly diverse and complex data collection methods. This unstable sampling environment has resulted in the generation of a large number of irregular monitoring data streams, posing significant challenges for related data analysis tasks. We have observed that irregular sequence sampling densities are uneven, containing randomly occurring dense and sparse intervals. This data imbalance tendency often leads to overfitting in the dense regions and underfitting in the sparse regions, ultimately impeding the representation performance of models. Conversely, the irregularity at the data level has limited impact on the deep semantics of sequences. Based on this observation, we propose a novel Non-isometric Alignment Inference Architecture (NAIA), which utilizes a multi-level semantic continuous representation structure based on inter-interval segmentation to learn representations of irregular sequences. This architecture efficiently extracts the latent features of irregular sequences. We evaluate the performance of NAIA on multiple datasets for downstream tasks and compare it with recent benchmark methods, demonstrating NAIA’s state-of-the-art performance results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Marlin, B.M., Kale, D.C., Khemani, R.G., Wetzel, R.C.: Unsupervised pattern discovery in electronic health care data using probabilistic clustering models. In: Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium, pp. 389–398 (2012)
Yadav, P., Steinbach, M., Kumar, V., Simon, G.: Mining electronic health records (EHRs) a survey. ACM Comput. Surv. (CSUR). 50, 1–40 (2018)
Schulz, M., Stattegger, K.: SPECTRUM: spectral analysis of unevenly spaced paleoclimatic time series. Comput. Geosci. 23, 929–945 (1997)
Dogariu, M., Ştefan, L.-D., Boteanu, B.A., Lamba, C., Kim, B., Ionescu, B.: Generation of realistic synthetic financial time-series. ACM Trans. Multimedia Comput. Commun. Appl. 18, 1–27 (2022)
Cheng, P., et al.: Asynchronous fault detection observer for 2-D markov jump systems. IEEE Trans Cybern. 52, 13623–13634 (2021)
Cheng, L.-F., Stück, D., Quisel, T., Foschini, L.: The Impact of Missing Data in User-Generated mHealth Time Series
Song, X., Sun, P., Song, S., Stojanovic, V.: Event-driven NN adaptive fixed-time control for nonlinear systems with guaranteed performance. J. Franklin Inst. 359, 4138–4159 (2022)
Fang, G., et al.: Up to 100x faster data-free knowledge distillation. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6597–6604 (2022)
Jiang, Y., Yin, S., Kaynak, O.: Performance supervised plant-wide process monitoring in industry 4.0: a roadmap. IEEE Open J. Ind. Electron. Soc. 2, 21–35 (2021)
Horn, M., Moor, M., Bock, C., Rieck, B., Borgwardt, K.: Set functions for time series. In: International Conference on Machine Learning, pp. 4353–4363. PMLR (2020)
Tan, Q., et al.: DATA-GRU: dual-attention time-aware gated recurrent unit for irregular multivariate time series. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 930–937 (2020)
Narayan Shukla, S., Marlin, B.M.: Multi-time attention networks for irregularly sampled time series. arXiv e-prints. arXiv-2101 (2021)
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. The MIT Press (2005). https://doi.org/10.7551/mitpress/3206.001.0001
Box, G.E.P., Jenkins, G.M., Reinsel, G.C., Ljung, G.M.: Time Series Analysis: Forecasting and Control. John Wiley & Sons (2015)
Kim, H.-G., Jang, G.-J., Choi, H.-J., Kim, M., Kim, Y.-W., Choi, J.: Recurrent neural networks with missing information imputation for medical examination data prediction. In: 2017 IEEE International Conference on Big Data and Smart Computing (BigComp), pp. 317–323. IEEE (2017)
Zhou, H., et al.: Informer: beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the AAAI conference on artificial intelligence, pp. 11106–11115 (2021)
Bonilla, E. V, Chai, K., Williams, C.: Multi-task Gaussian process prediction. Adv Neural Inf Process Syst. 20 (2007)
Shukla, S.N., Marlin, B.: Interpolation-prediction networks for irregularly sampled time series. In: International Conference on Learning Representations (2018)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30, (2017)
Trindade, A.: ElectricityLoadDiagrams20112014. Data Set, UCI Machine Learning Repository (2015)
Wang, K., et al.: Multiple convolutional neural networks for multivariate time series prediction. Neurocomputing 360, 107–119 (2019)
Zhang, J., Li, X., Tian, J., Luo, H., Yin, S.: An integrated multi-head dual sparse self-attention network for remaining useful life prediction. Reliab. Eng. Syst. Saf. 233, 109096 (2023)
Liu, S., et al.: Pyraformer: low-complexity pyramidal attention for long-range time series modeling and forecasting. In: International Conference on Learning Representations (2021)
Wu, H., Xu, J., Wang, J., Long, M.: Autoformer: decomposition transformers with auto-correlation for long-term series forecasting. Adv. Neural. Inf. Process. Syst. 34, 22419–22430 (2021)
Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., Jin, R.: Fedformer: frequency enhanced decomposed transformer for long-term series forecasting. In: International Conference on Machine Learning, pp. 27268–27286. PMLR (2022)
Fabius, O., van Amersfoort, J.R.: Variational recurrent auto-encoders. arXiv e-prints. arXiv-1412 (2014)
Su, Y., Zhao, Y., Niu, C., Liu, R., Sun, W., Pei, D.: Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2828–2837 (2019)
Abdulaal, A., Liu, Z., Lancewicki, T.: Practical approach to asynchronous multivariate time series anomaly detection and localization. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 2485–2494 (2021)
Hundman, K., Constantinou, V., Laporte, C., Colwell, I., Soderstrom, T.: Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 387–395 (2018)
Shen, L., Li, Z., Kwok, J.: Timeseries anomaly detection using temporal hierarchical one-class network. Adv. Neural. Inf. Process. Syst. 33, 13016–13026 (2020)
Zhang, Z.: Improved adam optimizer for deep neural networks. In: 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS), pp. 1–2. IEEE (2018)
Che, Z., Purushotham, S., Cho, K., Sontag, D., Liu, Y.: Recurrent neural networks for multivariate time series with missing values. Sci. Rep. 8, 6085 (2018)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yu, F., Li, S., Yu, W. (2024). Exploring Non-isometric Alignment Inference for Representation Learning of Irregular Sequences. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14452. Springer, Singapore. https://doi.org/10.1007/978-981-99-8076-5_24
Download citation
DOI: https://doi.org/10.1007/978-981-99-8076-5_24
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8075-8
Online ISBN: 978-981-99-8076-5
eBook Packages: Computer ScienceComputer Science (R0)