skip to main content
research-article

STAD-GAN: Unsupervised Anomaly Detection on Multivariate Time Series with Self-training Generative Adversarial Networks

Published: 27 February 2023 Publication History

Abstract

Anomaly detection on multivariate time series (MTS) is an important research topic in data mining, which has a wide range of applications in information technology, financial management, manufacturing system, and so on. However, the state-of-the-art unsupervised deep learning models for MTS anomaly detection are vulnerable to noise and have poor performance on the training data containing anomalies. In this article, we propose a novel Self-Training based Anomaly Detection with Generative Adversarial Network (GAN) model called STAD-GAN to address the practical challenge. The STAD-GAN model consists of a generator-discriminator structure for adversarial learning and a neural network classifier for anomaly classification. The generator is learned to capture the normal data distribution, and the discriminator is learned to amplify the reconstruction error of abnormal data for better recognition. The proposed model is optimized with a self-training teacher-student framework, where a teacher model generates reliable high-quality pseudo-labels to train a student model iteratively with a refined dataset so that the performance of the anomaly classifier can be gradually improved. Extensive experiments based on six open MTS datasets show that STAD-GAN is robust to noise and achieves significant performance improvement compared to the state-of-the-art.

References

[1]
Deepak Agarwal. 2005. An empirical bayes approach to detect anomalies in dynamic multidimensional arrays. In Proceedings of the 5th IEEE International Conference on Data Mining.IEEE Computer Society, 26–33.
[2]
Subutai Ahmad, Alexander Lavin, Scott Purdy, and Zuha Agha. 2017. Unsupervised real-time anomaly detection for streaming data. Neurocomputing 262, 11 (2017), 134–147.
[3]
Samet Akcay, Amir Atapour-Abarghouei, and Toby P. Breckon. 2019. GANomaly: Semi-supervised anomaly detection via adversarial training. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 11363 LNCS (2019), 622–637.
[4]
Layth C. Alwan and Harry V. Roberts. 1988. Time-series modeling for statistical process control. Journal of Business and Economic Statistics 6, 1 (1988), 87–95.
[5]
Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein GAN. arXiv abs/1701.07875. Retrieved from https://arxiv.org/abs/1701.07875.
[6]
Julien Audibert, Pietro Michiardi, Frédéric Guyard, Sébastien Marti, and M. A. Zuluaga. 2020. USAD: UnSupervised anomaly detection on multivariate time series. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
[7]
Markus M. Breunig, Hans-Peter Kriegel, Raymond T. Ng, and Jörg Sander. 2000. LOF: Identifying density-based local outliers. In Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data.Association for Computing Machinery, New York, NY, 93–104.
[8]
Raghavendra Chalapathy and Sanjay Chawla. 2019. Deep learning for anomaly detection: A survey. arXiv abs/1901.03407. Retrieved from https://arxiv.org/abs/1901.03407.
[9]
Jian-Hui Duan, Wenzhong Li, X. Zhang, and Sanglu Lu. 2022. Forecasting fine-grained city-scale cellular traffic with sparse crowdsourced measurements. Computer Networks 214, 9 (2022), 109156–109169. https://www.sciencedirect.com/science/article/pii/S1389128622002663.
[10]
E. Eskin. 2000. Anomaly detection over noisy data using learned probability distributions.In Proceedings of the 17th International Conference on Machine Learning.255–262.
[11]
Junyu Gao, Tao Han, Yuan Yuan, and Qi Wang. 2021. Domain-adaptive crowd counting via high-quality image translation and density reconstruction.IEEE Transactions on Neural Networks and Learning Systems (2021), 1–13.
[12]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Advances in Neural Information Processing Systems 27, 2 (2014), 2672–2680.
[13]
Manish Gupta, Jing Gao, Charu C. Aggarwal, and Jiawei Han. 2014. Outlier detection for temporal data. IEEE Transaction on Knowledge and Data Engineering 26, 9 (2014), 2250–2267.
[14]
Ilyass Haloui, Jayant Sen Gupta, and Vincent Feuillard. 2018. Anomaly detection with Wasserstein GAN. arXiv abs/1812.02463. Retrieved from https://arxiv.org/abs/1812.02463.
[15]
V. Hautamaki, I. Karkkainen, and P. Franti. 2004. Outlier detection using k-nearest neighbour graph. In Proceedings of the 17th International Conference on Pattern Recognition.430–433.
[16]
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soderstrom. 2018. Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.387–395.
[17]
Farzaneh Khoshnevisan and Zhewen Fan. 2019. RSM-GAN: A convolutional recurrent GAN for anomaly detection in contaminated seasonal multivariate time series. arXiv abs/1911.07104. Retrieved from https://arxiv.org/abs/1911.07104.
[18]
R. Laxhammar and G. Falkman. 2014. Online learning and sequential anomaly detection in trajectories. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 6 (2014), 1158–1173.
[19]
Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew P. Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, and Wenzhe Shi. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.105–114.
[20]
Dan Li, Dacheng Chen, Baihong Jin, Lei Shi, Jonathan Goh, and See Kiong Ng. 2019. MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 11730 LNCS (2019), 703–716.
[21]
F. Liu, K. Ting, and Z. Zhou. 2008. Isolation forest. In Proceedings of the 8th IEEE International Conference on Data Mining. 413–422.
[22]
Pankaj Malhotra, Lovekesh Vig, Gautam Shroff, and Puneet Agarwal. 2015. Long short term memory networks for anomaly detection in time series. In Proceedings of the 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning.89–94.
[23]
Daehyung Park, Yuuna Hoshi, and Charles C. Kemp. 2018. A Multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder. IEEE Robotics and Automation Letters 3, 3 (2018), 1544–1551.
[24]
Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. 2016. Generative adversarial text to image synthesis. In Proceedings of the International Conference on Machine Learning. 1060–1069.
[25]
Thomas Schlegl, Philipp Seeböck, Sebastian M. Waldstein, Ursula Schmidt-Erfurth, and Georg Langs. 2017. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In Proceedings of the Information Processing in Medical Imaging. Marc Niethammer, Martin Styner, Stephen Aylward, Hongtu Zhu, Ipek Oguz, Pew-Thian Yap, and Dinggang Shen (Eds.), Springer International Publishing, Cham, 146–157.
[26]
Alban Siffer, Pierre Alain Fouque, Alexandre Termier, and Christine Largouet. 2017. Anomaly detection in streams with extreme value theory. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.1067–1075.
[27]
Ya Su, Rong Liu, Youjian Zhao, Wei Sun, Chenhao Niu, and Dan Pei. 2019. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2828–2837.
[28]
Qizhe Xie, Minh Thang Luong, Eduard Hovy, and Quoc V. Le. 2020. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.10684–10695.
[29]
Haowen Xu, Wenxiao Chen, N. Zhao, Z. Li, Jiahao Bu, Zhihan Li, Y. Liu, Y. Zhao, D. Pei, Y. Feng, Jian Jhen Chen, Zhaogang Wang, and Honglin Qiao. 2018. Unsupervised anomaly detection via variational auto-encoder for seasonal KPIs in web applications. In Proceedings of the 2018 World Wide Web Conference.
[30]
H. Zare Moayedi and M. A. Masnadi-Shirazi. 2008. Arima model for network traffic prediction and anomaly detection. In Proceedings of the International Symposium on Information Technology. 1–6.
[31]
Chuxu Zhang, Dongjin Song, Yuncong Chen, Xinyang Feng, Cristian Lumezanu, Wei Cheng, Jingchao Ni, Bo Zong, Haifeng Chen, and Nitesh V. Chawla. 2019. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data. Proceedings of the AAAI Conference on Artificial Intelligence 33, 1 (2019), 1409–1416.
[32]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the International Conference on Computer Vision. 2242–2251.
[33]
Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen. 2018. Deep autoencoding Gaussian mixture model for unsupervised anomaly detection. In Proceedings of the 6th International Conference on Learning Representations. 1–19.
[34]
Barret Zoph, Golnaz Ghiasi, Tsung Yi Lin, Yin Cui, Hanxiao Liu, Ekin Cubuk, and Quoc V. Le. 2020. Rethinking pre-training and self-training. ArXiv. abs/2006.06882. Retrieved from https://arxiv.org/abs/2006.06882.

Cited By

View all
  • (2025)Landmark Block-Embedded Aggregation Autoencoder for Anomaly DetectionIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2024.349633255:2(1004-1019)Online publication date: Feb-2025
  • (2025)Detection of zero-day attacks via sample augmentation for the Internet of VehiclesVehicular Communications10.1016/j.vehcom.2025.10088752(100887)Online publication date: Apr-2025
  • (2024)Management Analysis Method of Multivariate Time Series Anomaly Detection in Financial Risk AssessmentJournal of Organizational and End User Computing10.4018/JOEUC.34209436:1(1-19)Online publication date: 16-Apr-2024
  • Show More Cited By

Index Terms

  1. STAD-GAN: Unsupervised Anomaly Detection on Multivariate Time Series with Self-training Generative Adversarial Networks

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Knowledge Discovery from Data
    ACM Transactions on Knowledge Discovery from Data  Volume 17, Issue 5
    June 2023
    386 pages
    ISSN:1556-4681
    EISSN:1556-472X
    DOI:10.1145/3583066
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 27 February 2023
    Online AM: 23 November 2022
    Accepted: 01 November 2022
    Revised: 22 August 2022
    Received: 06 January 2022
    Published in TKDD Volume 17, Issue 5

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Multivariate time series
    2. anomaly detection
    3. generative adversarial network
    4. self-training
    5. unsupervised learning

    Qualifiers

    • Research-article

    Funding Sources

    • National Natural Science Foundation of China
    • Key R&D Program of Jiangsu Province, China
    • Collaborative Innovation Center of Novel Software Technology and Industrialization
    • Sino-German Institutes of Social Computing

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)803
    • Downloads (Last 6 weeks)72
    Reflects downloads up to 12 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Landmark Block-Embedded Aggregation Autoencoder for Anomaly DetectionIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2024.349633255:2(1004-1019)Online publication date: Feb-2025
    • (2025)Detection of zero-day attacks via sample augmentation for the Internet of VehiclesVehicular Communications10.1016/j.vehcom.2025.10088752(100887)Online publication date: Apr-2025
    • (2024)Management Analysis Method of Multivariate Time Series Anomaly Detection in Financial Risk AssessmentJournal of Organizational and End User Computing10.4018/JOEUC.34209436:1(1-19)Online publication date: 16-Apr-2024
    • (2024)Curriculum Learning-Based Multivariate Time Series Anomaly Detection2024 35th Conference of Open Innovations Association (FRUCT)10.23919/FRUCT61870.2024.10516396(186-193)Online publication date: 24-Apr-2024
    • (2024)Anomaly detection in multivariate time series data using deep ensemble modelsPLOS ONE10.1371/journal.pone.030389019:6(e0303890)Online publication date: 6-Jun-2024
    • (2024)Variate Associated Domain Adaptation for Unsupervised Multivariate Time Series Anomaly DetectionACM Transactions on Knowledge Discovery from Data10.1145/366357318:8(1-24)Online publication date: 3-May-2024
    • (2024)Real- Time Anomaly Detection with Generative Adversarial Networks for Network Security Management2024 International Conference on Optimization Computing and Wireless Communication (ICOCWC)10.1109/ICOCWC60930.2024.10470902(1-6)Online publication date: 29-Jan-2024
    • (2024)TranBF: Deep Transformer Networks and Bayesian Filtering for Time Series Anomalous Signal Detection in Cyber-physical Systems2024 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME57554.2024.10687464(1-6)Online publication date: 15-Jul-2024
    • (2024)Treemil: A Multi-Instance Learning Framework for Time Series Anomaly Detection with Inexact SupervisionICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10447536(7510-7514)Online publication date: 14-Apr-2024
    • (2024)Targeted Detection of Anomalous Merchants on Integrated Payment Platforms via Multifaceted Transaction Representation Learning2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10825015(2170-2178)Online publication date: 15-Dec-2024
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media