Skip to main content
Springer Nature Link
Log in

DGTAD: decomposition GAN-based transformer for anomaly detection in multivariate time series data

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The advancement of the computer and information industry has led to the emergence of new demands for multivariate time series anomaly detection (MTSAD) models, namely, the necessity for unsupervised anomaly detection that is both efficient and accurate. However, long-term time series data typically encompass a multitude of intricate temporal pattern variations and noise. Consequently, accurately capturing anomalous patterns within such data and establishing precise and rapid anomaly detection models pose challenging problems. In this paper, we propose a decomposition GAN-based transformer for anomaly detection (DGTAD) in multivariate time series data. Specifically, DGTAD integrates a time series decomposition structure into the original transformer model, further decomposing the extracted global features into deep trend information and seasonal information. On this basis, we improve the attention mechanism, which uses decomposed time-dependent features to change the traditional focus of the transformer, enabling the model to reconstruct anomalies of different types in a targeted manner. This makes it difficult for anomalous data to adapt to these changes, thereby amplifying the anomalous features. Finally, by combining the GAN structure and using multiple generators from different perspectives, we alleviate the mode collapse issue, thereby enhancing the model’s generalizability. DGTAD has been validated on nine benchmark datasets, demonstrating significant performance improvements and thus proving its effectiveness in unsupervised anomaly detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Algorithm 2
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Availability of data and materials

The datasets used or analysed during the current study are available from the corresponding author on reasonable request.

Code Availability

Code availability not applicable.

References

  1. Pan T, Chen J, Xie J et al (2021) Deep feature generating network: A new method for intelligent fault detection of mechanical systems under class imbalance. IEEE Trans Ind Inform 17:6282–6293. https://doi.org/10.1109/TII.2020.3030967

    Article  Google Scholar 

  2. Fernando T, Gammulle H, Denman S et al (2021) Deep learning for medical anomaly detection - a survey. ACM Comput Surv 54:141:1-141:37. https://doi.org/10.1145/3464423

    Article  Google Scholar 

  3. Blázquez-García A, Conde A, Mori U et al (2021) A review on outlier/anomaly detection in time series data. ACM Comput Surv 54:56:1-56:33. https://doi.org/10.1145/3444690

    Article  Google Scholar 

  4. Breunig MM, Kriegel HP, Ng RT, et al (2000) LOF: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pp 93–10. https://doi.org/10.1145/342009.335388

  5. Miao X, Liu Y, Zhao H et al (2019) Distributed online one-class support vector machine for anomaly detection over networks. IEEE Trans Cybern 49:1475–148. https://doi.org/10.1109/TCYB.2018.2804940

    Article  Google Scholar 

  6. Ding Z, Fei M (2013) An anomaly detection approach based on isolation forest algorithm for streaming data using sliding window. IFAC Proc Vol 46:12–1. https://doi.org/10.3182/20130902-3-CN-3020.00044

    Article  Google Scholar 

  7. Lin J, He Y, Xu W et al (2023) Latent feature reconstruction for unsupervised anomaly detection. Appl Intell 53:23628–23640. https://doi.org/10.1007/s10489-023-04767-2

    Article  Google Scholar 

  8. Wang S, Cao J, Yu PS (2022) Deep learning for spatio-temporal data mining: A survey. IEEE Trans Knowl Data Eng 34(8):3681–3700. https://doi.org/10.1109/TKDE.2020.3025580

    Article  Google Scholar 

  9. Gui J, Sun Z, Wen Y et al (2023) A review on generative adversarial networks: Algorithms, theory, and applications. IEEE Trans Knowl Data Eng 35:3313–333. https://doi.org/10.1109/TKDE.2021.3130191

    Article  Google Scholar 

  10. Yang X, Li H, Feng X et al (2023) Variable-wise generative adversarial transformer in multivariate time series anomaly detection. Appl Intell 53:28745–2876. https://doi.org/10.1007/s10489-023-05029-x

    Article  Google Scholar 

  11. Lin XX, Lin P, Yeh EH (2021) Anomaly detection/prediction for the internet of things: State of the art and the future. IEEE Netw 35:212–218. https://doi.org/10.1109/MNET.001.1800552

    Article  Google Scholar 

  12. Kim C, Lee J, Kim R et al (2018) DeepNAP: Deep neural anomaly pre-detection in a semiconductor fab. Inf Sci 457–458:1–11. https://doi.org/10.1016/j.ins.2018.05.020

    Article  MathSciNet  Google Scholar 

  13. Munir M, Siddiqui SA, Dengel A et al (2019) DeepAnT: A deep learning approach for unsupervised anomaly detection in time series. IEEE Access 7:1991–200. https://doi.org/10.1109/ACCESS.2018.2886457

    Article  Google Scholar 

  14. Du H, Duan Z (2022) Finder: A novel approach of change point detection for multivariate time series. Appl Intell 52:2496–2509. https://doi.org/10.1007/s10489-021-02532-x

    Article  Google Scholar 

  15. Schmidl S, Wenig P, Papenbrock T (2022) Anomaly detection in time series: a comprehensive evaluation. Proc VLDB Endow 15:1779–1797. https://doi.org/10.14778/3538598.3538602

  16. Chow JK, Su Z, Wu J et al (2020) Anomaly detection of defects on concrete structures with the convolutional autoencoder. Adv Eng Inform 45:10110. https://doi.org/10.1016/j.aei.2020.101105

    Article  Google Scholar 

  17. Ge N, Weng X, Yang Q (2023) Unsupervised anomaly detection via two-dimensional singular value decomposition and subspace reconstruction for multivariate time series. Appl Intell 53:16813–16829. https://doi.org/10.1007/s10489-022-04337-y

    Article  Google Scholar 

  18. Du X, Yu J, Chu Z et al (2022) Graph autoencoder-based unsupervised outlier detection. Inf Sci 608:532–55. https://doi.org/10.1016/j.ins.2022.06.039

    Article  Google Scholar 

  19. Su Y, Zhao Y, Niu C, et al (2019) Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 2828–283. https://doi.org/10.1145/3292500.3330672

  20. Audibert J, Michiardi P, Guyard F, et al (2020) USAD: UnSupervised anomaly detection on multivariate time series. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 3395–340. https://doi.org/10.1145/3394486.3403392

  21. Li D, Chen D, Jin B, et al (2019) MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks. In: Tetko IV, Kůrková V, Karpov P, et al (eds) Artificial neural networks and machine learning – ICANN 2019: Text and Time Series, pp 703–716. https://doi.org/10.1007/978-3-030-30490-4_56

  22. Chen Y, Zhu H, Chen Z (2024) Multi-dgi: Multi-head pooling deep graph infomax for human activity recognition. Mob Netw Appl 1–12. https://doi.org/10.1007/s11036-024-02306-y

  23. Zhao H, Wang Y, Duan J, et al (2020) Multivariate time-series anomaly detection via graph attention network. In: 2020 IEEE international conference on data mining (ICDM), pp 841–85. https://doi.org/10.1109/ICDM50108.2020.00093

  24. Xu J, Wu H, Wang J, et al (2022) Anomaly transformer: Time series anomaly detection with association discrepancy. In: International conference on learning representations

  25. Li Y, Peng X, Zhang J et al (2023) DCT-GAN: Dilated convolutional transformer-based GAN for time series anomaly detection. IEEE Trans Knowl Data Eng 35:3632–3644. https://doi.org/10.1109/TKDE.2021.3130234

    Article  Google Scholar 

  26. Maru C, Brandherm B, Kobayashi I (2022) Verification of sparsity in the attention mechanism of transformer for anomaly detection in multivariate time series. In: 2022 IEEE international conference on big data (Big Data), pp 408–41. https://doi.org/10.1109/BigData55660.2022.10020675

  27. Tuli S, Casale G, Jennings NR (2022) TranAD: deep transformer networks for anomaly detection in multivariate time series data. Proc VLDB Endow 15:1201–1214. https://doi.org/10.14778/3514061.3514067

  28. Ma M, Han L, Zhou C (2023) BTAD: A binary transformer deep neural network model for anomaly detection in multivariate time series data. Adv Eng Inform 56:10194. https://doi.org/10.1016/j.aei.2023.101949

    Article  Google Scholar 

  29. Zhang C, Zhou T, Wen Q, et al (2022) Tfad: A decomposition time series anomaly detection architecture with time-frequency analysis. In: Proceedings of the 31st ACM international conference on information & knowledge management, pp 2497–250. https://doi.org/10.1145/3511808.3557470

  30. Chen S, Chang CI, Li X (2022) Component decomposition analysis for hyperspectral anomaly detection. IEEE Trans Geosci Remote Sens 60:1–2. https://doi.org/10.1109/TGRS.2021.3117765

    Article  Google Scholar 

  31. Wu H, Xu J, Wang J, et al (2021) Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In: Advances in neural information processing systems, pp 22419–22430

  32. Zhou T, Ma Z, Wen Q, et al (2022) Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In: Proceedings of the 39th international conference on machine learning, pp 27268–27286

  33. Sen R, Yu HF, Dhillon IS (2019) Think globally, act locally: A deep neural network approach to high-dimensional time series forecasting. Adv Neural Inf Process Syst 32

  34. Han J, Pei J, Tong H (2022) Data mining: concepts and techniques

  35. Zeng P, Hu G, Zhou X et al (2023) Seformer: a long sequence time-series forecasting model based on binary position encoding and information transfer regularization. Appl Intell 53:15747–15771. https://doi.org/10.1007/s10489-022-04263-z

  36. Ahmad S, Lavin A, Purdy S et al (2017) Unsupervised real-time anomaly detection for streaming data. Neurocomputing 262:134–147. https://doi.org/10.1016/j.neucom.2017.04.070

    Article  Google Scholar 

  37. Hundman K, Constantinou V, Laporte C, et al (2018) Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 387–39. https://doi.org/10.1145/3219819.3219845

  38. Mathur AP, Tippenhauer NO (2016) SWaT: a water treatment testbed for research and training on ICS security. In: 2016 International workshop on cyber-physical systems for smart water networks (CySWater), pp 31–36. https://doi.org/10.1109/CySWater.2016.7469060

  39. Ahmed CM, Palleti VR, Mathur AP (2017) WADI: a water distribution testbed for research in the design of secure cyber physical systems. In: Proceedings of the 3rd international workshop on cyber-physical systems for smart water networks, pp 25–2. https://doi.org/10.1145/3055366.3055375

  40. Nedelkoski S, Bogatinovski J, Mandapati AK, et al (2020) Multi-source distributed system data for AI-powered analytics. In: Service-oriented and cloud computing, pp 161–17. https://doi.org/10.1007/978-3-030-44769-4_13

  41. Moody G, Mark R (2001) The impact of the MIT-BIH arrhythmia database. IEEE Eng Med Biol Mag 20:45–50. https://doi.org/10.1109/51.932724

    Article  Google Scholar 

  42. Keogh E, Dutta RT, Naik U, et al (2021) Multi-dataset time-series anomaly detection competition. In: ACM SIGKDD international conference on knowledge discovery and data mining, https://compete.hexagon-ml.com/practice/competition/39/

  43. Nakamura T, Imamura M, Mercer R, et al (2020) MERLIN: Parameter-free discovery of arbitrary length anomalies in massive time series archives. In: 2020 IEEE international conference on data mining (ICDM), pp 1190–1195. https://doi.org/10.1109/ICDM50108.2020.00147

  44. Li S, Yu J, Lu Y et al (2024) Self-supervised enhanced denoising diffusion for anomaly detection. Inf Sci 669:120612. https://doi.org/10.1016/j.ins.2024.120612

    Article  Google Scholar 

  45. Huang S, Liu Y, Fung C et al (2020) Hitanomaly: Hierarchical transformers for anomaly detection in system log. IEEE Trans Netw Serv Manag 17:2064–207. https://doi.org/10.1109/TNSM.2020.3034647

    Article  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China (Grant Nos.62262064).

Author information

Authors and Affiliations

  1. School of Software, Xinjiang University, Urumqi, 830091, China

    Zixin Chen, Jiong Yu & Qiyin Tan

  2. School of Information Science and Engineering, Xinjiang University, Urumqi, 830046, China

    Jiong Yu, Shu Li & XuSheng Du

Authors
  1. Zixin Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jiong Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Qiyin Tan
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Shu Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. XuSheng Du
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Jiong Yu.

Ethics declarations

Conflicts of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Z., Yu, J., Tan, Q. et al. DGTAD: decomposition GAN-based transformer for anomaly detection in multivariate time series data. Appl Intell 54, 13038–13056 (2024). https://doi.org/10.1007/s10489-024-05693-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05693-7

Keywords