skip to main content
research-article

Variate Associated Domain Adaptation for Unsupervised Multivariate Time Series Anomaly Detection

Published: 26 July 2024 Publication History

Abstract

Multivariate Time Series Anomaly Detection (MTS-AD) is crucial for the effective management and maintenance of devices in complex systems, such as server clusters, spacecrafts, and financial systems, and so on. However, upgrade or cross-platform deployment of these devices will introduce the issue of cross-domain distribution shift, which leads to the prototypical problem of domain adaptation for MTS-AD. Compared with general domain adaptation problems, MTS-AD domain adaptation presents two peculiar challenges: (1) the dimensions of data from the source domain and the target domain are usually different, so alignment without losing any information is necessary; and (2) the association between different variates plays a vital role in the MTS-AD task, which is overlooked by traditional domain adaptation approaches. Aiming at addressing the above issues, we propose a Variate Associated Domain Adaptation Method Combined with a Graph Deviation Network (VANDA) for MTS-AD, which includes two major contributions. First, we characterize the intra-domain variate associations of the source domain by a graph deviation network (GDN), which can share parameters across domains without dimension alignment. Second, we propose a sliding similarity to measure the inter-domain variate associations and perform joint training by minimizing the optimal transport distance between source and target data for transferring variate associations across domains. VANDA achieves domain adaptation by transferring both variate associations and GDN parameters from the source domain to the target domain. We construct two pairs of MTS-AD datasets from existing MTS-AD data and combine three domain adaptation strategies with six MTS-AD backbones as the benchmark methods for experimental evaluation and comparison. Extensive experiments demonstrate the effectiveness of our approach, which outperforms the benchmark methods, and significantly improves the AD performance of the target domain by effectively utilizing the source domain knowledge.

References

[1]
Chuadhry Mujeeb Ahmed, Venkata Reddy Palleti, and Aditya P. Mathur. 2017. WADI: A water distribution testbed for research in the design of secure cyber physical systems. In Proceedings of the 3rd International Workshop on Cyber-Physical Systems for Smart Water Networks (CySWATER@CPSWeek ’17). (April 21, 2017). Panagiotis Tsakalides and Baltasar Beferull-Lozano (Eds.). ACM, 25–28. DOI:
[2]
Julien Audibert, Pietro Michiardi, Frédéric Guyard, Sébastien Marti, and Maria A Zuluaga. 2020. USAD: Unsupervised anomaly detection on multivariate time series. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 3395–3404.
[3]
Sriram Baireddy, Sundip R. Desai, James L. Mathieson, Richard H. Foster, Moses W. Chan, Mary L. Comer, and Edward J. Delp. 2021. Spacecraft time-series anomaly detection using transfer learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1951–1960.
[4]
Jiang Bian, Xiaolong Hui, Shiying Sun, Xiaoguang Zhao, and Min Tan. 2019. A novel and efficient CVAE-GAN-based approach with informative manifold for semi-supervised anomaly detection. IEEE Access 7 (2019), 88903–88916.
[5]
Markus M. Breunig, Hans-Peter Kriegel, Raymond T. Ng, and Jörg Sander. 2000. LOF: Identifying density-based local outliers. In Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data. (May 16–18, 2000). Weidong Chen, Jeffrey F. Naughton, and Philip A. Bernstein (Eds.). ACM, 93–104. DOI:
[6]
Ruichu Cai, Jiawei Chen, Zijian Li, Wei Chen, Keli Zhang, Junjian Ye, Zhuozhang Li, Xiaoyan Yang, and Zhenjie Zhang. 2021. Time series domain adaptation via sparse associative structure alignment. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, 6859–6867.
[7]
Ningjiang Chen, Huan Tu, Xiaoyan Duan, Liangqing Hu, and Chengxiang Guo. 2023. Semisupervised anomaly detection of multivariate time series based on a variational autoencoder. Applied Intelligence 53, 5 (2023), 6074–6098.
[8]
Wenchao Chen, Long Tian, Bo Chen, Liang Dai, Zhibin Duan, and Mingyuan Zhou. 2022. Deep variational graph convolutional recurrent network for multivariate time series anomaly detection. In Proceedings of the International Conference on Machine Learning. PMLR, 3621–3633.
[9]
Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio. 2015. A recurrent latent variable model for sequential data. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 28, 2980–2988.
[10]
Quanyu Dai, Xiao-Ming Wu, Jiaren Xiao, Xiao Shen, and Dan Wang. 2022. Graph transfer learning via adversarial domain adaptation with graph convolution. IEEE Transactions on Knowledge and Data Engineering 35, 5 (2022), 4908–4922.
[11]
Ailin Deng and Bryan Hooi. 2021. Graph neural network-based anomaly detection in multivariate time series. In Proceedings of the 35th AAAI Conference on Artificial Intelligence. AAAI Press, 4027–4035. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16523
[12]
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Francois Laviolette, Mario Marchand, and Victor S. Lempitsky. 2016. Domain-adversarial training of neural networks. Journal of Machine Learning Research 17 (2016), 59:1–59:35. Retrieved from http://jmlr.org/papers/v17/15-239.html
[13]
Jonathan Goh, Sridhar Adepu, Khurum Nazir Junejo, and Aditya Mathur. 2016. A dataset to support research in the design of secure water treatment systems. In Proceedings of the Critical Information Infrastructures Security - 11th International Conference (CRITIS ’16). (October 10–12, 2016). Revised Selected Papers:opi: (Lecture Notes in Computer Science, Vol. 10242). Grigore M. Havarneanu, Roberto Setola, Hypatia Nassopoulos, and Stephen D. Wolthusen (Eds.). Springer, 88–99. DOI:
[14]
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Söderström. 2018. Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’18). (August 19–23, 2018). Yike Guo and Faisal Farooq (Eds.). ACM, 387–395. DOI:
[15]
Amir Hussein and Hazem Hajj. 2022. Domain adaptation with representation learning and nonlinear relation for time series. ACM Transactions on Internet of Things 3, 2 (2022), 1–26.
[16]
Siwon Kim, Kukjin Choi, Hyun-Soo Choi, Byunghan Lee, and Sungroh Yoon. 2021. Towards a rigorous evaluation of time-series anomaly detection. In Proceedings of the 36th AAAI Conference on Artificial Intelligence. AAAI Press, 7194–7201. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/download/20680/20439
[17]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (ICLR) (Poster). Retrieved from https://arxiv.org/pdf/1412.6980
[18]
Zhihan Li, Youjian Zhao, Jiaqi Han, Ya Su, Rui Jiao, Xidao Wen, and Dan Pei. 2021. Multivariate time series anomaly detection and interpretation using hierarchical inter-metric and temporal embedding. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’21). (August 14–18, 2021). Feida Zhu, Beng Chin Ooi, and Chunyan Miao (Eds.). ACM, 3220–3230. DOI:
[19]
Guansong Pang, Chunhua Shen, and Anton van den Hengel. 2019. Deep anomaly detection with deviation networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’19). (August 4–8, 2019). Ankur Teredesai, Vipin Kumar, Ying Li, Rómer Rosales, Evimaria Terzi, and George Karypis (Eds.). ACM, 353–362. DOI:
[20]
Daehyung Park, Yuuna Hoshi, and Charles C. Kemp. 2018. A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder. IEEE Robotics and Automation Letters 3, 2 (2018), 1544–1551. DOI:
[21]
Gabriel Peyré and Marco Cuturi. 2019. Computational optimal transport: With applications to data science. Foundations and Trends® in Machine Learning 11, 5–6 (2019), 355–607.
[22]
Sanjay Purushotham, Wilka Carvalho, Tanachat Nilanon, and Yan Liu. 2016. Variational recurrent adversarial deep domain adaptation. In Proceedings of the International Conference on Learning Representations.
[23]
Artem Rozantsev, Mathieu Salzmann, and Pascal Fua. 2018. Beyond sharing weights for deep domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 4 (2018), 801–814.
[24]
Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Emmanuel Müller, and Marius Kloft. 2018. Deep one-class classification. In Proceedings of the International Conference on Machine Learning. PMLR, 4393–4402.
[25]
Bernhard Schölkopf, John C. Platt, John Shawe-Taylor, Alexander J. Smola, and Robert C. Williamson. 2001. Estimating the support of a high-dimensional distribution. Neural Computation 13, 7 (2001), 1443–1471. DOI:
[26]
Lifeng Shen, Zhuocong Li, and James T. Kwok. 2020. Timeseries anomaly detection using temporal hierarchical one-class network. In Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS ’20), Vol. 33, 13016–13026. (December 6–12, 2020), virtual, Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). Retrieved from https://proceedings.neurips.cc/paper/2020/hash/97e401a02082021fd24957f852e0e475-Abstract.html
[27]
Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. 2019. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’19). (August 4–8, 2019). Ankur Teredesai, Vipin Kumar, Ying Li, Rómer Rosales, Evimaria Terzi, and George Karypis (Eds.). ACM, 2828–2837. DOI:
[28]
Baochen Sun and Kate Saenko. 2016. Deep CORAL: Correlation alignment for deep domain adaptation. In Proceedings of the Computer Vision - ECCV 2016 Workshops. (October 8–10 and 15–16, 2016), Part III (Lecture Notes in Computer Science, Vol. 9915). Gang Hua and Hervé Jégou (Eds.), 443–450. DOI:
[29]
David M. J. Tax and Robert P. W. Duin. 2004. Support vector data description. Machine Learning 54, 1 (2004), 45–66.
[30]
Shreshth Tuli, Giuliano Casale, and Nicholas R. Jennings. 2022. TranAD: Deep transformer networks for anomaly detection in multivariate time series data. Proceedings of the VLDB Endowment, 15, 6 (2022), 1201–1214.
[31]
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’17). (July 21–26, 2017). IEEE Computer Society, 2962–2971. DOI:
[32]
Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. 2014. Deep domain confusion: Maximizing for domain invariance. Retrieved from http://arxiv.org/abs/1412.3474
[33]
Vincent Vercruyssen, Wannes Meert, and Jesse Davis. 2017. Transfer learning for time series anomaly detection. In Proceedings of the Workshop and Tutorial on Interactive Adaptive Learning@ ECMLPKDD 2017, Vol. 1924. CEUR Workshop Proceedings, 27–37.
[34]
Vercruyssen Vincent, Meert Wannes, and Davis Jesse. 2020. Transfer learning for anomaly detection through localized and unsupervised instance selection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, 6054–6061.
[35]
Zengmao Wang, Bo Du, and Yuhong Guo. 2019. Domain adaptation with neural embedding matching. IEEE Transactions on Neural Networks and Learning Systems 31, 7 (2019), 2387–2397.
[36]
Garrett Wilson and Diane J. Cook. 2020. A survey of unsupervised deep domain adaptation. ACM Transactions on Intelligent Systems and Technology (TIST) 11, 5 (2020), 1–46.
[37]
Garrett Wilson, Janardhan Rao Doppa, and Diane J. Cook. 2020. Multi-source deep domain adaptation with weak supervision for time-series sensor data. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 1768–1778.
[38]
Haowen Xu, Wenxiao Chen, Nengwen Zhao, Zeyan Li, Jiahao Bu, Zhihan Li, Ying Liu, Youjian Zhao, Dan Pei, Yang Feng, and J. Chen. 2018. Unsupervised anomaly detection via variational auto-encoder for seasonal KPIs in web applications. In Proceedings of the 2018 World Wide Web Conference, 187–196.
[39]
Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. 2021. Anomaly transformer: Time series anomaly detection with association discrepancy. Retrieved from https://openreview.net/forum?id=LzQQ89U1qm_
[40]
Renjun Xu, Pelen Liu, Liyan Wang, Chao Chen, and Jindong Wang. 2020. Reliable weighted optimal transport for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4394–4403.
[41]
Jianfei Yang, Han Zou, Yuxun Zhou, Zhaoyang Zeng, and Lihua Xie. 2020. Mind the discriminability: Asymmetric adversarial domain adaptation. In Proceedings of the Computer Vision–ECCV 2020: 16th European Conference. (August 23–28, 2020), Part XXIV 16. Springer, 589–606.
[42]
Jiying Zhang, Xi Xiao, Long-Kai Huang, Yu Rong, and Yatao Bian. 2022b. Fine-tuning graph neural networks via graph topology induced optimal transport. In Proceedings of the 31st International Joint Conference on Artificial Intelligence, 3730–3736. Retrieved from
[43]
Yuxin Zhang, Jindong Wang, Yiqiang Chen, Han Yu, and Tao Qin. 2022a. Adaptive memory networks with self-supervised learning for unsupervised anomaly detection. IEEE Transactions on Knowledge and Data Engineering 35, 12 (2022), 12068–12080.
[44]
Zhijie Zhang, Wenzhong Li, Wangxiang Ding, Linming Zhang, Qingning Lu, Peng Hu, Tong Gui, and Sanglu Lu. 2023. STAD-GAN: Unsupervised anomaly detection on multivariate time series with self-training generative adversarial networks. ACM Transactions on Knowledge Discovery from Data 17, 5 (2023), 1–18.
[45]
Fuzhen Zhuang, Xiaohu Cheng, Ping Luo, Sinno Jialin Pan, and Qing He. 2015. Supervised representation learning: Transfer learning with deep autoencoders. In Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI ’15). (July 25–31, 2015). Qiang Yang and Michael J. Wooldridge (Eds.). AAAI Press, 4119–4125. Retrieved from http://ijcai.org/Abstract/15/578
[46]
Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Dae-ki Cho, and Haifeng Chen. 2018. Deep autoencoding Gaussian mixture model for unsupervised anomaly detection. In Proceedings of the 6th International Conference on Learning Representations (ICLR ’18). (April 30–May 3, 2018), Conference Track Proceedings. OpenReview.net. Retrieved from https://openreview.net/forum?id=BJJLHbb0-

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data
ACM Transactions on Knowledge Discovery from Data  Volume 18, Issue 8
September 2024
700 pages
EISSN:1556-472X
DOI:10.1145/3613713
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 July 2024
Online AM: 03 May 2024
Accepted: 23 April 2024
Revised: 29 February 2024
Received: 10 February 2023
Published in TKDD Volume 18, Issue 8

Check for updates

Author Tags

  1. anomaly detection
  2. multivariate time series
  3. domain adaptation
  4. optimal transport

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 568
    Total Downloads
  • Downloads (Last 12 months)568
  • Downloads (Last 6 weeks)55
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media