Abstract
Multivariate time series representation learning employs unsupervised tasks to extract meaningful representations from time series data, enabling their application in diverse downstream tasks. However, despite the promising advancements in contrastive learning-based representation learning, the study of task-related feature learning is still in its early stages. This gap exists because current unified representation learning frameworks lack the ability to effectively disentangle task-related features. To address this limitation, we propose DisT, a novel contrastive learning-based method for efficient task-related feature learning in time series representation. DisT disentangles task-related features by incorporating feature network structure learning and contrastive sample pair selection. Specifically, DisT incorporates a feature decoupling module, which prioritizes global features for time series classification tasks, while emphasizing periodic and seasonal features for forecasting tasks. Additionally, DisT leverages contrastive loss and task-related feature loss to adaptively select data augmentation methods, preserving task-relevant shared information between positive samples across different data and tasks. Experimental results on various multivariate time-series datasets including classification and forecasting tasks show that DisT achieves state-of-the-art performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bagnall, A., et al.: The UEA multivariate time series classification archive. arXiv preprint arXiv:1811.00075 (2018)
Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018)
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
Cleveland, R.B., Cleveland, W.S., McRae, J.E., Terpenning, I.: STL: a seasonal-trend decomposition. J. Off. Stat 6(1), 3–73 (1990)
Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)
Eldele, E., et al.: Time-series representation learning via temporal and contextual contrasting (2021)
Franceschi, J.-Y., Dieuleveut, A., Jaggi, M.: Unsupervised scalable representation learning for multivariate time series. Adv. Neural Inf. Process. Syst. 32 (2019)
Harutyunyan, H., Khachatrian, H., Kale, D.C., Ver Steeg, G., Galstyan, A.: Multitask learning and benchmarking with clinical time series data. Scientific Data 6(1), 96 (2019)
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738 (2020)
Lai, G., Chang, W.-C., Yang, Y., Liu, H.: Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 95–104 (2018)
Le Guennec, A., Malinowski, S., Tavenard, R.: Data augmentation for time series classification using convolutional neural networks. In: ECML/PKDD Workshop on Advanced Analytics and Learning on Temporal Data (2016)
Li, S., et al.: Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Adv. Neural Inf. Process. Syst. 32 (2019)
Li, Y., Hu, G., Wang, Y., Hospedales, T., Robertson, N.M., Yang, Y.: Dada: differentiable automatic data augmentation. arXiv preprint arXiv:2003.03780 (2020)
Luo, D., et al.: Time series contrastive learning with information-aware augmentations. Proc. AAAI Conf. Artif. Intell. 37, 4534–4542 (2023)
van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)
Theiler, J., Eubank, S., Longtin, A., Galdrikian, B., Farmer, J.D.: Testing for nonlinearity in time series: the method of surrogate data. Physica D: Nonl. Phenom. 58(1–4), 77–94 (1992)
Tian, Y., Sun, C., Poole, B., Krishnan, D., Schmid, C., Isola, P.: What makes for good views for contrastive learning? Adv. Neural. Inf. Process. Syst. 33, 6827–6839 (2020)
Tonekaboni, S., Eytan, D., Goldenberg, A.: Unsupervised representation learning for time series with temporal neighborhood coding. In: International Conference on Learning Representations
Tonekaboni, S., Eytan, D., Goldenberg, A.: Unsupervised representation learning for time series with temporal neighborhood coding. arXiv preprint arXiv:2106.00750 (2021)
Van der Maaten, L., Hinton, G.: Visualizing data using t-sne. J. Mach. Learn. Res. 9(11) (2008)
Wang, H., Guo, X., Deng, Z.-H., Lu, Y.: Rethinking minimal sufficient representation in contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16041–16050 (2022)
Wang, Z., Xovee, X., Zhang, W., Trajcevski, G., Zhong, T., Zhou, F.: Learning latent seasonal-trend representations for time series forecasting. Adv. Neural. Inf. Process. Syst. 35, 38775–38787 (2022)
Woo, G., Liu, C., Sahoo, D., Kumar, A., Hoi, S.: Cost: contrastive learning of disentangled seasonal-trend representations for time series forecasting. arXiv preprint arXiv:2202.01575 (2022)
Yu, H., Yang, H., Sano, A.: Leaves: learning views for time-series data in contrastive learning. arXiv preprint arXiv:2210.07340 (2022)
Yue, Z., et al.: Ts2vec: towards universal representation of time series. Proc. AAAI Conf. Artif. Intell. 36, 8980–8987 (2022)
Zeng, A., Chen, M., Zhang, L., Qiang, X.: Are transformers effective for time series forecasting? Proc. AAAI Conf. Artif. Intell. 37, 11121–11128 (2023)
Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., Eickhoff, C.: A transformer-based framework for multivariate time series representation learning. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2114–2124 (2021)
Zhou, H., et al.: Informer: beyond efficient transformer for long sequence time-series forecasting. Proc. AAAI Conf. Artif. Intell. 35, 11106–11115 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Hou, L., Pan, L., Guo, Y., Li, C., Zhang, L. (2024). Learning Disentangled Task-Related Representation for Time Series. In: Yang, DN., Xie, X., Tseng, V.S., Pei, J., Huang, JW., Lin, J.CW. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2024. Lecture Notes in Computer Science(), vol 14650. Springer, Singapore. https://doi.org/10.1007/978-981-97-2266-2_18
Download citation
DOI: https://doi.org/10.1007/978-981-97-2266-2_18
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2265-5
Online ISBN: 978-981-97-2266-2
eBook Packages: Computer ScienceComputer Science (R0)