Abstract
Due to the intricate dynamics of multivariate time series in cyber-physical system, unsupervised anomaly detection has always been a research hotspot. Common methods are mainly based on reducing reconstruction error or maximizing estimated probability for normal data, however, both of them may be sensitive to particular fluctuations in data. Meanwhile, these methods tend to model temporal dependency or spatial correlation individually, which is insufficient to detect diverse anomalies. In this paper, we propose an error-restricted framework with variance estimation, namely Spatial-Temporal Anomaly Transformer (S-TAR), which can provide a corresponding confidence for each reconstruction. First, it presents Error-Restricted Probability (ERP) loss by restricting the reconstruction error and its estimated probability skillfully, further improving the capability to distinguish outliers from normal data. Second, we adopt Spatial-Temporal Transformer with distinct attention modules to detect diverse anomalies. Extensive experiments on five real-world datasets are conducted, the results show that our method is superior to existing state-of-the-art approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdulaal, A., Liu, Z., Lancewicki, T.: Practical approach to asynchronous multivariate time series anomaly detection and localization, pp. 2485–2494. Association for Computing Machinery (2021)
Ahmed, C.M., Palleti, V.R., Mathur, A.P.: Wadi: a water distribution testbed for research in the design of secure cyber physical systems, pp. 25–28. Association for Computing Machinery (2017)
Audibert, J., Michiardi, P., Guyard, F., Marti, S., Zuluaga, M.A.: Usad: unsupervised anomaly detection on multivariate time series. In: Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 3395–3404 (2020)
Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling (2018)
Bai, Z.Z., Golub, G.H., Ng, M.K.: Hermitian and skew-hermitian splitting methods for non-hermitian positive definite linear systems. SIAM J. Matrix Anal. Appl. 24(3), 603–626 (2003)
Dai, E., Chen, J.: Graph-augmented normalizing flows for anomaly detection of multiple time series. In: Proceedings of the 10th International Conference on Learning Representations (2022)
Deng, A., Hooi, B.: Graph neural network-based anomaly detection in multivariate time series. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 4027–4035 (2021)
Feng, C., Tian, P.: Time series anomaly detection for cyber-physical systems via neural system identification and bayesian filtering. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 2858–2867 (2021)
Goh, J., Adepu, S., Tan, M., Lee, Z.S.: Anomaly detection in cyber physical systems using recurrent neural networks. In: Proceedings of the 18th International Symposium on High Assurance Systems Engineering (HASE), pp. 140–145 (2017)
Hundman, K., Constantinou, V., Laporte, C., Colwell, I., Soderstrom, T.: Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding, pp. 387–395. Association for Computing Machinery (2018)
Kim, S., Choi, K., Choi, H.S., Lee, B., Yoon, S.: Towards a rigorous evaluation of time-series anomaly detection (2022)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes (2013)
Lee, J., Bagheri, B., Kao, H.A.: A cyber-physical systems architecture for industry 4.0-based manufacturing systems. Manufact. Lett. 3, 18–23 (2015)
Li, Z., et al.: Multivariate time series anomaly detection and interpretation using hierarchical inter-metric and temporal embedding. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 3220–3230. Association for Computing Machinery (2021)
Liu, F.T., Ting, K.M., Zhou, Z.H.: Isolation forest. In: Proceedings of the Eighth IEEE International Conference on Data Mining (ICDM)m pp. 413–422 (2008)
Mathur, A.P., Tippenhauer, N.O.: SWaT: a water treatment testbed for research and training on ics security. In: Proceedings of 2016 International Workshop on Cyber-physical Systems for Smart Water Networks, pp. 31–36 (2016)
Omar, S., Ngadi, A., Jebur, H.H.: Article: machine learning techniques for anomaly detection: an overview. Inter. J. Comput. Appli. 79(2), 33–41 (2013)
Pang, G., Shen, C., Cao, L., Hengel, A.V.D.: Deep learning for anomaly detection: a review, vol. 54(2) (mar 2021)
Park, D., Hoshi, Y., Kemp, C.C.: A multimodal anomaly detector for robot-assisted feeding using an lstm-based variational autoencoder. IEEE Robot. Automat. Lett. 3(3), 1544–1551 (2018)
Rezende, D., Mohamed, S.: Variational inference with normalizing flows. In: Proceedings of the 32nd International Conference on Machine Learning, pp. 1530–1538. PMLR, Lille, France (2015)
Shen, L., Yu, Z., Ma, Q., Kwok, J.T.: Time series anomaly detection with multiresolution ensemble decoding. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 9567–9575 (2021)
Su, Y., Zhao, Y., Niu, C., Liu, R., Sun, W., Pei, D.: Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In: Proceedings of the 25th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 2828–2837 (2019)
Tax, D.M.J., Duin, R.P.W.: Support vector data description, vol. 54, pp. 45–66 (2004)
Tuli, S., Casale, G., Jennings, N.R.: TranAD: deep transformer networks for anomaly detection in multivariate time series data. In: Proceedings of Very Large Data Bases, vol. 15, pp. 1201–1214 (2022)
Vaswani, A., et al.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, vol. 30, pp. 6000–6010. Curran Associates, Inc. (2017)
Zhou, B., Liu, S.H., Hooi, B., Cheng, X., Ye, J.: BeatGAN: anomalous rhythm detection using adversarially generated time series. In: International Joint Conference on Artificial Intelligence (2019)
Zong, B., et al.: Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In: International Conference on Learning Representations (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Feng, Y., Zhang, W., Sun, H., Jiang, W. (2024). Spatial-Temporal Transformer with Error-Restricted Variance Estimation for Time Series Anomaly Detection. In: Yang, DN., Xie, X., Tseng, V.S., Pei, J., Huang, JW., Lin, J.CW. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2024. Lecture Notes in Computer Science(), vol 14645. Springer, Singapore. https://doi.org/10.1007/978-981-97-2242-6_1
Download citation
DOI: https://doi.org/10.1007/978-981-97-2242-6_1
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2241-9
Online ISBN: 978-981-97-2242-6
eBook Packages: Computer ScienceComputer Science (R0)