Abstract
With the proliferation of the Internet of Things (IoT), there is an abundance of data available to humans. However, the monitoring environments for data collection are becoming increasingly diverse, leading to the occurrence of concept drift in the collected data. Concept drift refers to the phenomenon where the distribution of data changes over time, making it challenging for prediction models trained on historical data to adapt to the changing distribution. Previous research has primarily focused on predicting or compensating for distributions with fixed durations in Euclidean space to mitigate non-stationarity. However, we have observed that concept drift often occurs at different time scales, and detecting them using fixed scales has inherent limitations. Based on this observation, we propose a Graph-Guided Latent Variable Target Inference network that maps current data and variable duration query targets onto a graph neural network in latent space. We apply self-attention transformations to the representations and correlations on the graph in the dimensions of time, features, and query targets. The model updates its parameters based on these non-Euclidean correlation patterns, enabling the graph to evolve towards the direction of the query targets and obtain an evolved latent distribution. Finally, the decoder generates a prediction data stream regarding the query targets based on the evolved latent distribution. The experiments were conducted on five datasets, where our proposed method was compared against the five most advanced baselines. The findings demonstrated a substantial advantage in prediction performance provided by our approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zhang, C., Cui, L., Yu, S., James, J.Q.: A communication-efficient federated learning scheme for iot-based traffic forecasting. IEEE Internet Things J. 9(14), 11918–11931 (2021)
Shengdong, M., Zhengxian, X., Yixiang, T.: Intelligent traffic control system based on cloud computing and big data mining. IEEE Trans. Industr. Inf. 15(12), 6583–6592 (2019)
Han, J., Liu, H., Zhu, H., Xiong, H., Dou, D.: Joint air quality and weather prediction based on multiadversarial spatiotemporal networks. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence (2021)
Salinas, D., Flunkert, V., Gasthaus, J., Januschowski, T.: DeepAR: probabilistic forecasting with autoregressive recurrent networks. Int. J. Forecast. 36(3), 1181–1191 (2020)
Agrahari, S., Singh, A.K.: Concept drift detection in data stream mining: a literature review. J. King Saud Univ.-Comput. Inf. Sci. 34(10), 9523–9540 (2022)
Casado, F.E., Lema, D., Criado, M.F., Iglesias, R., Regueiro, C.V., Barro, S.: Concept drift detection and adaptation for federated and continual learning. Multimedia Tools Appl. 1–23 (2022)
Zenisek, J., Holzinger, F., Affenzeller, M.: Machine learning based concept drift detection for predictive maintenance. Comput. Ind. Eng. 137, 106031 (2019)
Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., Zhang, G.: Learning under concept drift: a review. IEEE Trans. Knowl. Data Eng. 31(12), 2346–2363 (2018)
Passalis, N., Tefas, A., Kanniainen, J., Gabbouj, M., Iosifidis, A.: Deep adaptive input normalization for time series forecasting. IEEE Trans. Neural Networks Learn. Syst. 31(9), 3760–3765 (2019)
Kim, T., Kim, J., Tae, Y., Park, C., Choi, J.H., Choo, J.: Reversible instance normalization for accurate time-series forecasting against distribution shift. In: International Conference on Learning Representations (2021)
Abu-Rayash, A., Dincer, I.: Analysis of the electricity demand trends amidst the COVID-19 coronavirus pandemic. Energy Res. Soc. Sci. 68, 101682 (2020)
Ren, S., Liao, B., Zhu, W., Li, K.: Knowledgemaximized ensemble algorithm for different types of concept drift. Inf. Sci. 430, 261–281 (2018)
Žliobaitė, I., Pechenizkiy, M., Gama, J.: An ˙overview of concept drift applications. In: Japkowicz, N., Stefanowski, J. (eds.) Big Data Analysis: New Algorithms for a New Society. Studies in Big Data, vol. 16, pp. 91–114. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-26989-4_4
Khamassi, I., Sayed-Mouchaweh, M., Hammami, M., Ghedira, K.: Discussion and review on evolving data streams and concept drift adapting. Evol. Syst. 9(1), 1–23 (2018)
Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., Wortman Vaughan, J.: A theory of learning from different domains. Mach. Learn. 79(1), 151–175 (2010)
Ganin, Y., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(1), 2096–2030 (2016)
Hoffman, J., Darrell, T., Saenko, K.: Continuous manifold based adaptation for evolving visual domains. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 867–874 (2014)
Mancini, M., Rota Bulo, S., Caputo, B., Ricci, E.: AdaGraph: unifying predictive and continuous domain adaptation through graphs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6568–6577 (2019)
Wang, H., He, H., Katabi, D.: Continuously indexed domain adaptation. In: International Conference on Machine Learning, pp. 9898–9907 (2020)
Du, Y., et al.: AdaRNN: adaptive learning and forecasting of time series. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 402–411 (2021)
Kim, T., Kim, J., Tae, Y., Park, C., Choi, J.-H., Choo, J.: Reversible instance normalization for accurate timeseries forecasting against distribution shift. In: International Conference on Learning Representations (2022)
Dhaka, A.K., Catalina, A., Welandawe, M., Andersen, M.R., Huggins, J., Vehtari, A.: Challenges and opportunities in high dimensional variational inference. Adv. Neural. Inf. Process. Syst. 34, 7787–7798 (2021)
Chen, J., Zhu, J., Teh, Y.W., Zhang, T.: Stochastic expectation maximization with variance reduction. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Zorich, V.A., Paniagua, O.: Mathematical Analysis II, vol. 220. Springer, Heidelberg (2016)
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. stat 1050(20), 10–48550 (2017)
Ertugrul, F.O.: Forecasting electricity load by a novel recurrent extreme learning machines approach. Int. J. Electr. Power Energy Syst. 78, 429–435 (2016)
Abdulaal, A., Liu, Z., Lancewicki, T.: Practical approach to asynchronous multivariate time series anomaly detection and localization. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 2485–2494 (2021)
Su, Y., Zhao, Y., Niu, C., Liu, R., Sun, W., Pei, D.: Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2828–2837 (2019)
Hundman, K., Constantinou, V., Laporte, C., Colwell, I., Soderstrom, T.: Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 387–395 (2018)
Shen, L., Li, Z., Kwok, J.: Timeseries anomaly detection using temporal hierarchical one-class network. Adv. Neural. Inf. Process. Syst. 33, 13016–13026 (2020)
Sun, Y., Pfahringer, B., Gomes, H.M., Bifet, A.: SOKNL: a novel way of integrating K-nearest neighbours with adaptive random forest regression for data streams. Data Min. Knowl. Disc. 36(5), 2006–2032 (2022)
Zhao, P., Cai, L.-W., Zhou, Z.-H.: Handling concept drift via model reuse. Mach. Learn. 109(3), 533–568 (2020)
Rubanova, Y., Chen, R.T.Q., Duvenaud, D.K.: Latent ordinary differential equations for irregularly-sampled time series. In: Advances in Neural Information Processing Systems, vol. 32, pp. 5320–5330. Curran Associates, Inc. (2019)
Zhou, H., et al.: Informer: Beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35(12), pp. 11106–11115 (2021)
Wu, H., Xu, J., Wang, J., Long, M.: AutoFormer: decomposition transformers with auto-correlation for long-term series forecasting. Adv. Neural. Inf. Process. Syst. 34, 22419–22430 (2021)
Kenton, J.D.M.W.C., Toutanova, L.K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, vol. 1, p. 2 (2019)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yu, F., Li, S., Yu, W. (2024). Graph-Guided Latent Variable Target Inference for Mitigating Concept Drift in Time Series Forecasting. In: Liu, F., Sadanandan, A.A., Pham, D.N., Mursanto, P., Lukose, D. (eds) PRICAI 2023: Trends in Artificial Intelligence. PRICAI 2023. Lecture Notes in Computer Science(), vol 14327. Springer, Singapore. https://doi.org/10.1007/978-981-99-7025-4_31
Download citation
DOI: https://doi.org/10.1007/978-981-99-7025-4_31
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-7024-7
Online ISBN: 978-981-99-7025-4
eBook Packages: Computer ScienceComputer Science (R0)