Skip to main content
Springer Nature Link
Log in

A cross-layered cluster embedding learning network with regularization for multivariate time series anomaly detection

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The devices deployed across diverse industrial scenarios have generated significant network traffic related to time. The system’s irregular operation could result in substantial bad influence. Anomaly detection technologies utilized for identifying possible non-standard behaviours are paramount; furthermore, multivariate time series exhibit complex dependencies besides temporal correlation. However, most previous methods merely consider the temporal and variable correlation of time series data, neglecting the distance metrics among the sequences, leading to a deficiency in the model’s anomaly detection ability. We propose a multivariate time series anomaly detection model based on the encoder–decoder architecture (CCER-ED). The model considers the similarity measure between temporal subsequences and designs a multi-scale feature embedding module for leveraging more interrelated properties. Moreover, the interrelations among sensors are explicitly learned using a manifold regularization graph structure. On this basis, an improved data fusion approach based on a multi-head self-attention mechanism is designed for capturing global feature representation, effectively integrating various aspects of information. Evaluations using the real-world datasets SWAT and WADI and performance analysis show that the proposed approach achieves improvement over the baselines in the recall and F1-score of anomaly detection performance at 9.3% and 8.5% (maximum), respectively, outperforming other existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

Authors do not have permission to share the datasets used.

References

  1. Cai J, Liang W, Li X, Li K, Gui Z, Khan MK (2023) GTxChain: a secure IoT smart blockchain architecture based on graph neural network. IEEE Internet of Things J 1–14

  2. Xu Z, Liang W, Li K-C, Xu J, Zomaya AY, Zhang J (2021) A time-sensitive token-based anonymous authentication and dynamic group key agreement scheme for industry 5.0. IEEE Trans Ind Inform 18(10):7118–7127

    Article  Google Scholar 

  3. Erhan L, Ndubuaku M, Di Mauro M et al (2021) Smart anomaly detection in sensor systems: a multi-perspective review. Inf Fus 67:64–79

    Article  Google Scholar 

  4. Ge M, Bangui H, Buhnova B (2018) Big data for internet of things: a survey. Futur Gener Comput Syst 87:601–614

    Article  Google Scholar 

  5. Ding C, Sun S, Zhao J (2023) MST-GAT: a multimodal spatial-temporal graph attention network for time series anomaly detection. Inf Fus 89:527–536

    Article  Google Scholar 

  6. Kong F, Li J, Jiang B et al (2021) Integrated generative model for industrial anomaly detection via bidirectional LSTM and attention mechanism. IEEE Trans Ind Inf 19(1):541–550

    Article  Google Scholar 

  7. Li Y, Liang W, Xie K, Zhang D, Xie S, Li K (2023) Lightnestle: quick and accurate neural sequential tensor completion via meta learning. In: IEEE Infocom

  8. Fahim M, Fraz K, Sillitti A (2020) TSI: Time series to imaging based model for detecting anomalous energy consumption in smart buildings. Inf Sci 523:1–13

    Article  Google Scholar 

  9. Bertalanič B, Meža M, Fortuna C (2022) Resource-aware time series imaging classification for wireless link layer anomalies. IEEE Trans Neural Netw Learn Syst 1–13

  10. Siffer A, Fouque P-A, Termier A et al (2017) Anomaly detection in streams with extreme value theory. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 1067–1075

  11. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv (CSUR) 41(3):1–58

    Article  Google Scholar 

  12. Cook A, Mısırlı G, Fan Z (2019) Anomaly detection for IoT time-series data: a survey. IEEE Internet Things J 7(7):6481–6494

    Article  Google Scholar 

  13. Audibert J, Michiardi P, Guyard F et al (2020) USAD: unsupervised anomaly detection on multivariate time series. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 3395–3404

  14. Sun J et al (2019) Univariate time series classification using information geometry. Pattern Recogn 95:24–35

    Article  Google Scholar 

  15. Yang X, Zhang Z, Cui R (2022) TimeCLR: a self-supervised contrastive learning framework for univariate time series representation. Knowl-Based Syst 245:108606

    Article  Google Scholar 

  16. Kim D, Baek J-G (2022) Bagging ensemble-based novel data generation method for univariate time series forecasting. Expert Syst Appl 203:117366

    Article  Google Scholar 

  17. Oreshkin BN, Carpov D, Chapados N, Bengio Y (2019) N-beats: neural basis expansion analysis for interpretable time series forecasting. In: International Conference on Learning Representations, pp 1–31

  18. Wu H, Shi S (2021) Real-time anomaly detection in gas sensor streaming data. Int J Embedded Syst 14(1):81–88

    Article  MathSciNet  Google Scholar 

  19. Williams BM, Hoel LA (2003) Modeling and forecasting vehicular traffic flow as a seasonal arima process: theoretical basis and empirical results. J Transp Eng 129(6):664–672

    Article  Google Scholar 

  20. Wang W, Bao J, Li T (2021) Bound smoothing based time series anomaly detection using multiple similarity measures. J Intell Manuf 32:1711–1727

    Article  Google Scholar 

  21. Aggarwal CC (2017) An introduction to outlier analysis. Springer, Berlin, pp 1–34

    Google Scholar 

  22. Ji Z, Wang Y, Yan K et al (2022) A space-embedding strategy for anomaly detection in multivariate time series. Expert Syst Appl 206:117892

    Article  Google Scholar 

  23. Li Y, Peng X, Zhang J et al (2021) DCT-GAN: dilated convolutional transformer-based GAN for time series anomaly detection. IEEE Trans Knowl Data Eng

  24. Zhou X, Hu Y, Wu J, Liang W, Ma J, Jin Q (2022) Distribution bias aware collaborative generative adversarial network for imbalanced deep learning in industrial IoT. IEEE Trans Ind Inf 19(1):570–580

    Article  Google Scholar 

  25. Wang D, Gan J, Mao J et al (2023) Forecasting power demand in china with a CNN-LSTM model including multimodal information. Energy 263:126012

    Article  Google Scholar 

  26. Yoon J, Jarrett D, Schaar M (2019) Time-series generative adversarial networks. In: Advances in Neural Information Processing Systems, vol 32

  27. Zhang X, Gan H (2023) STF-Net: an improved depth network based on spatio-temporal data fusion for pm2.5 concentration prediction. Future Gener Comput Syst 144:37–49

    Article  Google Scholar 

  28. Breunig M, Kriegel H, Ng R et al (2000) LOF: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp 93–104

  29. Kriegel H, Schubert M, Zimek A (2008) Angle-based outlier detection in high-dimensional data. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 444–452

  30. He Z, Xu X, Deng S (2003) Discovering cluster-based local outliers. Pattern Recogn Lett 24(9–10):1641–1650

    Article  Google Scholar 

  31. Keogh E, Lin J, Fu A (2005) Hot sax: efficiently finding the most unusual time series subsequence. In: Fifth IEEE International Conference on Data Mining (ICDM’05), p 8

  32. Zhou Y, Ren H, Li Z et al (2022) Anomaly detection based on a granular Markov model. Expert Syst Appl 187:115744

    Article  Google Scholar 

  33. Kaur R, Gabrijelčič D (2022) Behavior segmentation of electricity consumption patterns: a cluster analytical approach. Knowl-Based Syst 251:109236

    Article  Google Scholar 

  34. Scarselli F, Gori M, Tsoi A et al (2008) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80

    Article  Google Scholar 

  35. Liang W, Li Y, Xie K, Zhang D, Li K-C, Souri A, Li K (2022) Spatial-temporal aware inductive graph neural network for C-ITS data recovery. IEEE Trans Intell Transp Syst

  36. Li Y, Yu R, Shahabi C, Liu Y (2018) Diffusion convolutional recurrent neural network: data-driven traffic forecasting. In: Proceedings of the International Conference on Learning Representations (ICLR)

  37. Diao C, Zhang D, Liang W, Li K-C, Hong Y, Gaudiot J-L (2022) A novel spatial-temporal multi-scale alignment graph neural network security model for vehicles prediction. IEEE Trans Intell Transp Syst 24(1):904–914

    Article  Google Scholar 

  38. Wei X, Zhang Y, Wei Y et al (2021) Metro passenger-flow representation via dynamic mode decomposition and its application. IEEE Trans Neural Netw Learn Syst 34:157–170

    Article  Google Scholar 

  39. Martínez-Galarza J, Bianco F, Crake D et al (2021) A method for finding anomalous astronomical light curves and their analogues. Mon Not R Astron Soc 508(4):5734–5756

    Article  Google Scholar 

  40. Li L, Yan J, Wang H et al (2020) Anomaly detection of time series with smoothness-inducing sequential variational auto-encoder. IEEE Trans Neural Netw Learn Syst 32(3):1177–1191

    Article  MathSciNet  Google Scholar 

  41. Gao X, Hu W, Guo Z (2020) Exploring structure-adaptive graph learning for robust semi-supervised classification. In: 2020 IEEE International Conference on Multimedia and Expo (ICME), pp 1–6

  42. Egilmez HE, Pavez E, Ortega A (2017) Graph learning from data under Laplacian and structural constraints. IEEE J Sel Top Signal Process 11(6):825–841

    Article  Google Scholar 

  43. Long J, Liang W, Li K-C, Wei Y, Marino MD (2022) A regularized cross-layer ladder network for intrusion detection in industrial internet of things. IEEE Trans Ind Inf 19(2):1747–1755

    Article  Google Scholar 

  44. Ma M, Han L, Zhou C (2023) BTAD: a binary transformer deep neural network model for anomaly detection in multivariate time series data. Adv Eng Inform 56:101949

    Article  Google Scholar 

  45. Chen Z, Chen D, Zhang X et al (2021) Learning graph structures with transformer for multivariate time-series anomaly detection in IoT. IEEE Internet Things J 9(12):9179–9189

    Article  MathSciNet  Google Scholar 

  46. Maddison C, Tarlow D, Minka T (2014) A* sampling. In: NeurIPS, pp 3086–3094

  47. Jang E, Gu S, Poole B (2016) Categorical reparameterization with Gumbel-Softmax. In: International Conference on Learning Representations, pp 1–13

  48. Deng A, Hooi B (2021) Graph neural network-based anomaly detection in multivariate time series. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 35, pp 4027–4035

  49. Mathur A, Tippenhauer N (2016) Swat: a water treatment testbed for research and training on ICS security. In: 2016 International Workshop on Cyber-physical Systems for Smart Water Networks, pp 31–36

  50. Ahmed C, Palleti V, Mathur A (2017) WADI: a water distribution testbed for research in the design of secure cyber physical systems. In: Proceedings of the 3rd International Workshop on Cyber Physical Systems for Smart Water Networks, pp 25–28

  51. Hundman K, Constantinou V, Laporte C et al (2018) Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 387–395

  52. Goh J, Adepu S, Junejo K et al (2017) A dataset to support research in the design of secure water treatment systems. In: 11th International Conference, CRITIS, pp 88–99

Download references

Funding

This work was partially supported by the Excellent Youth Funding of Hunan Provincial Education Department under Grant 23B0082, the Natural Science Foundation of Hunan Province under Grant 2021JJ30455, the National Key Research and Development Program of China under Grant 2021YFA1000600, the National Natural Science Foundation of China under Grant 62072170, the Science and Technology Project of Department of Communications of Hunan Provincial under Grant 202101, the Key Research and Development Program of Hunan Province under Grant 2022GK2015, the Hunan Provincial Natural Science Foundation of China under Grant 2021JJ30141, and the Open Research Fund of Hunan Provincial Key Laboratory of Network Investigational Technology under grant 2020WLZC001.

Author information

Authors and Affiliations

  1. The College of Information Science and Engineering, Hunan Normal University, Changsha, 410081, China

    Jing Long, Cuiting Luo, Ruxin Chen & Jianping Yu

  2. Department of Computer Science and Information Engineering, Providence University, Taichung, 43301, Taiwan

    Kuan-Ching Li

Authors
  1. Jing Long
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Cuiting Luo
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Ruxin Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jianping Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Kuan-Ching Li
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

JL was involved in Methodology, Writing-Original draft preparation, Software, and Supervision; CL contributed to Conceptualization, Methodology, Writing-Original draft preparation, Resources, and Software; RC was involved in Methodology, Data curation, and Resources; JY contributed to Resources, Software, and Validation; K-CL was involved in Methodology, Data curation, Validation, Visualization, Supervision, and Writing-Reviewing and Editing;

Corresponding authors

Correspondence to Cuiting Luo or Kuan-Ching Li.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Long, J., Luo, C., Chen, R. et al. A cross-layered cluster embedding learning network with regularization for multivariate time series anomaly detection. J Supercomput 80, 10444–10468 (2024). https://doi.org/10.1007/s11227-023-05833-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05833-9

Keywords