Abstract
Reconstruction-based algorithms offer state-of-the-art performance in multivariate time series anomaly detection. But as always: there is no single best algorithm. To find the optimal solution, one has to compare different methods and tune their hyperparameters. This paper introduces a lightweight modular benchmarking framework for data scientists and researchers in the field. The framework can be easily set up and automatically create a visual summary of the relevant performance indicators and automatically selected examples to give insight into the behavior of the model and aid during the development.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Code and documetnation of our framework: https://github.com/Arn-BAuA/TimeSeriesAEBenchmarkSuite.
- 2.
Detailed information on the interfaces is provided in the documentation.
- 3.
Our demo video: https://youtu.be/QPpVbOcZ_xU.
References
Cometml experiment tracking software. https://www.comet.com/site/. Accessed 14 Apr 2023
Mlflow experiment tracking software. https://github.com/KDD-OpenSource/DeepADoTS. Accessed 14 Apr 2023
Mlflow experiment tracking software. https://mlflow.org/. Accessed 14 Apr 2023
Mtad repository. https://github.com/OpsPAI/MTAD. Accessed 14 Apr 2023
Neptune.ai experiment tracking software. https://neptune.ai/. Accessed 14 Apr 2023
Orion repository. https://github.com/sintel-dev/Orion. Accessed 14 Apr 2023
pythae repository. https://github.com/clementchadebec/benchmark_VAE. Accessed 14 Apr 2023
Tensorboard experiment tracking software. https://www.tensorflow.org/tensorboard. Accessed 14 Apr 2023
Wandb experiment tracking software. https://wandb.ai/site. Accessed 14 Apr 2023
Akcay, S., Atapour-Abarghouei, A., Breckon, T.P.: : Ganomaly: semi-supervised anomaly detection via adversarial training. In: Computer Vision-ACCV 2018:14th Asian Conference on Computer Vision, Perth, Australia, December 2–6 14. (2019)
Böing, B., Klüttermann, S., Müller, E.: : Post-robustifying deep anomaly detection ensembles by model selection (2022)
Kaminskyi, D., Li, B., Müller, E.: : Reconstruction-based unsupervised drift detection over multivariate streaming data (2022)
Su, Y., Zhao, Y., Niu, C., Liu, R., Sun, W., Pei, D.: Robust anomaly detection for multivariate time series through stochastic recurrent neural network (2019)
Zong, B., et al.: Deep autoencoding gaussian mixture model for unsupervised anomaly detection (2018)
Acknowledgements
This work was supported by the Observatory on Artificial Intelligence in Work and Society as part of the Policy Lab Digital, Work & Society of the Federal Ministry for Labour and Social Affairs (BMAS).
This work was supported by the Research Center Trustworthy Data Science and Security, an institution of the University Alliance Ruhr.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Baudzus, A., Li, B., Jadid, A., Müller, E. (2023). The Good, The Bad, and The Average: Benchmarking of Reconstruction Based Multivariate Time Series Anomaly Detection. In: De Francisci Morales, G., Perlich, C., Ruchansky, N., Kourtellis, N., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Applied Data Science and Demo Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14175. Springer, Cham. https://doi.org/10.1007/978-3-031-43430-3_30
Download citation
DOI: https://doi.org/10.1007/978-3-031-43430-3_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43429-7
Online ISBN: 978-3-031-43430-3
eBook Packages: Computer ScienceComputer Science (R0)
