Abstract
The advancement of sensor technology has made it possible to use more sensors to monitor industrial systems, resulting in a large amount of irregular, unlabeled time-series data. Consequently, a large volume of irregular and unlabeled time series data is produced. Learning appropriate representations for those series is a very important but challenging task. This paper presents a self-supervised representation learning model SRLI (Self-supervised Representation Learning for Irregularities). We use T-LSTM to construct the irregularity encoder block. Based on this, we design three data augmentation methods. First, the raw time-series data are transformed into different yet correlated views. Second, we propose a contrasting module to learn robust representations. Lastly, to further learn discriminative representations, we reconstruct the series and try to get the imputation values of the unobserved positions. Rather than in a two-stage manner, our framework can generate the instance-level representation for ISMTS directly. Experiments show that this model has good performance on multiple data sets.
This work was supported by the National Key R &D Program of China under Grant No. 2020YFB1710200 and Heilongjiang Key R &D Program of China under Grant No. GA23A915.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bagnall, A., et al.: The UEA multivariate time series classification archive, 2018. arXiv:1811.00075 [cs, stat], October 2018. arXiv: 1811.00075
Baytas, I.M., Xiao, C., Zhang, X., Wang, F., Jain, A.K., Zhou, J.: Patient subtyping via time-aware LSTM networks. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’17, pp. 65–74. Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3097983.3097997
Cao, W., Wang, D., Li, J., Zhou, H., Li, L., Li, Y.: Brits: bidirectional recurrent imputation for time series. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 31. Curran Associates, Inc. (2018)
Eldele, E., et al: Time-series representation learning via temporal and contextual contrasting. In: Zhou, Z. (ed.) Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19–27 August 2021, pp. 2352–2359. ijcai.org (2021). https://doi.org/10.24963/ijcai.2021/324
Franceschi, J.Y., Dieuleveut, A., Jaggi, M.: Unsupervised scalable representation learning for multivariate time series. In: Advances in Neural Information Processing Systems, vol. 32. Curran Associates, Inc. (2019)
Gidaris, S., Singh, P., Komodakis, N.: Unsupervised representation learning by predicting image rotations. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=S1v4N2l0-
Kreindler, D.M., Lumsden, C.J.: The effects of the irregular sample and missing data in time series analysis. In: Nonlinear Dynamics, Psychology, and Life Sciences (2006)
Lipton, Z.C., Kale, D.C., Elkan, C., Wetzel, R.: Learning to diagnose with LSTM recurrent neural networks, March 2017. https://doi.org/10.48550/arXiv.1511.03677, arXiv:1511.03677 [cs]
Miao, X., Wu, Y., Wang, J., Gao, Y., Mao, X., Yin, J.: Generative semi-supervised learning for multivariate time series imputation. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, 2–9 February 2021, pp. 8983–8991. AAAI Press (2021). https://ojs.aaai.org/index.php/AAAI/article/view/17086
Noroozi, M., Favaro, P.: Unsupervised learning of visual representations by solving jigsaw puzzles. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 69–84. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_5
van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)
Sun, C., Hong, S., Song, M., Li, H.: A review of deep learning methods for irregularly sampled medical time series data. arXiv preprint arXiv:2010.12493 (2020)
Tonekaboni, S., Eytan, D., Goldenberg, A.: Unsupervised representation learning for time series with temporal neighborhood coding. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, 3–7 May 2021. OpenReview.net (2021). https://openreview.net/forum?id=8qDwejCuCN
Wang, J., Jiao, J., Liu, Y.-H.: Self-supervised video representation learning by pace prediction. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12362, pp. 504–521. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58520-4_30
Yang, X., Zhang, Z., Cui, R.: TimeCLR: a self-supervised contrastive learning framework for univariate time series representation. Knowl.-Based Syst. 245, 108606 (2022). https://doi.org/10.1016/j.knosys.2022.108606
Yoon, J., Zame, W.R., van der Schaar, M.: Estimating missing data in temporal data streams using multi-directional recurrent neural networks. IEEE Trans. Biomed. Eng. 66(5), 1477–1490 (2019). https://doi.org/10.1109/TBME.2018.2874712
Yue, Z., et al.: Ts2vec: towards universal representation of time series. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 8980–8987 (2022)
Zhang, R., Isola, P., Efros, A.A.: Colorful image colorization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 649–666. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_40
Zhang, X., Zeman, M., Tsiligkaridis, T., Zitnik, M.: Graph-guided network for irregularly sampled multivariate time series. In: The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, 25–29 April 2022. OpenReview.net (2022)
Zhou, H., et al.: Informer: beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 12, pp. 11106–11115, May 2021. https://doi.org/10.1609/aaai.v35i12.17325, https://ojs.aaai.org/index.php/AAAI/article/view/17325
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhang, H., Zhang, X., Han, Q., Lu, D. (2024). SRLI: Handling Irregular Time Series with a Novel Self-supervised Model Based on Contrastive Learning. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1967. Springer, Singapore. https://doi.org/10.1007/978-981-99-8178-6_25
Download citation
DOI: https://doi.org/10.1007/978-981-99-8178-6_25
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8177-9
Online ISBN: 978-981-99-8178-6
eBook Packages: Computer ScienceComputer Science (R0)