Abstract
Internet of Things (IoT) is network based on information carriers such as the Internet and traditional telecommunications networks, so that all ordinary physical objects that can be independently addressed can be interconnected. In the face of the IoT produces a large of time series data, which is very necessary to detect anomaly data. Transformer has proven to be a powerful tool in several areas, but still has some limitations, such as the prediction accuracy is not high enough. As the dominant trend of multivariate time series in different scenarios becomes increasingly evident, it is particularly important to accurately capture the spatio-temporal features between them. To address these issues, we propose Dynamic Graph transFormer (DGFormer), an effective Dynamic Graph Transformer based Anomaly Detection Model for IoT Time Series. We first use Transformer with anomaly attention mechanism to extract time features. Then, a dynamic relationship embedding strategy is proposed to capture spatio-temporal features dynamically and learn the adjacency matrix adaptively. Besides, each layer of GNN is soft clustered by Diffpooling. Finally, in order to further improve the detection performance of model, we integrate the traditional autoregressive linear model with the nonlinear neural network in parallel. The experimental results show that the proposed model achieves the highest F1-score on three public IoT datasets, and the F1-score is improved by 19.3% on average.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Renjie, W., Eamonn, J.K.: Current time series anomaly detection benchmarks are flawed and are creating the illusion of progress. IEEE Trans. Knowl. Data Eng. 35(3), 2421–2429 (2021)
Liang, W., Huang, W., Long, J., et al.: Deep reinforcement learning for resource protection and real-time detection in IoT environment. IEEE Internet Things J. 7(7), 6392–6401 (2020)
Muhammad, S.: Fog computing and its role in the internet of things: concept, security and privacy issues. Int. J. Comput. Appl. 180(32), 7–9 (2018)
Xin, R., Chen, P., Zhao, Z.: CausalRCA: causal inference based precise fine-grained root cause localization for microservice applications. J. Syst. Softw. 203, 111724 (2023). https://doi.org/10.1016/j.jss.2023.111724
Peng, C., et al.: Effectively detecting operational anomalies in large-scale IoT data infrastructures by using a GAN-based predictive model. Comput. J. 65(11), 2909–2925 (2022)
Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. (CSUR) 41(3), 1–58 (2009)
Zhang, R., Chen, J., Song, Y., Shan, W., Chen, P., Xia, Y.: An effective transformation-encoding-attention framework for multivariate time series anomaly detection in IoT environment. Mob. Netw. Appl. 1–13 (2023). https://doi.org/10.1007/s11036-023-02204-9
Tang, M., Fu, X., Wu, H., Huang, Q., Zhao, Q.: Traffic flow anomaly detection based on robust ridge regression with particle swarm optimization algorithm. Math. Prob. Eng. 2020, 1–10 (2020)
Venkatesan, R., et al.: Hyperspectral image features classification using deep learning recurrent neural networks. J. Med. Syst. (2019). https://doi.org/10.1007/s10916-019-1347-9
Wu, Y., Dai, H.N., Tang, H.: Graph neural networks for anomaly detection in industrial internet of things. IEEE Internet Things J. 9(12), 9214–9231 (2021). https://doi.org/10.1109/JIOT.2021.3094295
Kahya, E., Theodossiou, P.: Predicting corporate finacial distress: a time-series CUSUM methodology’. Rev. Quant. Finan. Account. 13(4), 323–345 (1996)
Janacek, G.: Time series analysis forecasting and control. J. Time 31(4), 303 (2010)
Chen, Y., Wang, S., Zhao, Q., Sun, G.: Detection of multivariate geochemical anomalies using the bat-optimized isolation forest and bat-optimized elliptic envelope models. J. Earth Sci. 32(2), 415–426 (2021)
Lai, G., Chang, W.C., Yang, Y., Liu, H.: Modeling long-and short-term temporal patterns with deep neural networks. In: International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 95–104. ACM (2018)
Song, Y., Xin, R., Chen, P., Zhang, R., Chen, J., Zhao, Z.: Identifying performance anomalies in fluctuating cloud environments: a robust correlative-GNN-based explainable approach. Future Gener. Comput. Syst. 145, 77–86 (2023)
Park, D., Hoshi, Y., Kemp, C.C.: A multimodal anomaly detector for robot-assisted feeding using an lstm-based variational autoencoder. IEEE Rob. Autom. Lett. 3(3), 1544–1551 (2018)
Fazle, K., Somshubra, M., Houshang, D.: Insights into lstm fully convolutional networks for time series classification. IEEE Access 7, 67718–67725 (2019)
Zhang, X., Gao, Y., Lin, J., et al.: TapNet: multivariate time series classification with attentional prototypical network. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 6845–6852 (2020)
Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45(11), 2673–2681 (1997)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014). https://doi.org/10.3115/v1/D14-1179
Shen, L., Li, Z., Kwok, J.: Timeseries anomaly detection using temporal hierarchical one-class network. Adv. Neural Inf. Process. Syst. 33, 13016–13026 (2020)
Mehdi, M., Bing, X., et al.: Generative adversarial networks. Commun. ACM 63, 139–144 (2020)
Qi, S., Chen, J., Chen, P., Wen, P., Niu, X., Xu, L.: An efficient GAN-based predictive framework for multivariate time series anomaly prediction in cloud data centers. J. Supercomput. 1–26 (2023). https://doi.org/10.1007/s11227-023-05534-3
Xavier, B., et al.: A generalization of Transformer networks to graphs. DLG-AAAI (2020). https://doi.org/10.48550/arXiv.2012.09699
Shao, P., He, J., Li, G., Zhang, D., Tao, J.: Hierarchical graph attention network for temporal knowledge graph reasoning. Neurocomputing 550, 126390 (2023)
Devin, K., et al.: Rethinking graph transformers with spectral attention. In: NeurIPS (2021). https://doi.org/10.48550/arXiv.2106.03893
Chen, D., et al.: A trainable optimal transport embedding for feature aggregation and its relationship to attention. In: ICLR (2021). https://doi.org/10.48550/arXiv.2006.12065
Pan, Y., et al.: A novel approach to scheduling workflows upon cloud resources with fluctuating performance. MONET 25(2), 690–700 (2020)
Chen, P., Xia, Y., Pang, S., Li, J.: A probabilistic model for performance analysis of cloud infrastructures. Concurr. Comput. Pract. Exp. 27(17), 4784–4796 (2015)
Christopher, M.B., et al.: Pattern Recognition and Machine Learning. Springer, New York (2006)
Ahmed, A., Zhuanghua, L., Tomer, L.: Practical approach to asynchronous multivariate time series anomaly detection and localization. In: KDD, pp. 2485–2494 (2021)
Ya, S., Wei, S., et al.: Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In: SIGKDD Explorations, pp. 2828–2837 (2019)
Houssam, Z., Manon, R., Bruno, L., et al.: Adversarially learned anomaly detection. In: IEEE International Conference on Data Mining (ICDM) (2018). https://doi.org/10.1109/ICDM.2018.00088
Bernhard, S., et al.: Support vector method for novelty detection. Adv. Neural Inf. Process. Syst. (1999)
Liu, Y., Li, Z., Zhou, C., et al.: Generative adversarial active learning for unsupervised outlier detection. IEEE Trans. Knowl. Data Eng. 32(8), 1517–1528 (2019). https://doi.org/10.1109/TKDE.2019.2905606
Julien, A., Pietro, M., Frédéric, G., Sébastien, M., Maria A.Z.: USAD: unsupervised anomaly detection on multivariate time series. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3395–3404 (2020)
Martin, S., Ralf, S., Hermann, N.: LSTM neural networks for language modeling. In: Interspeech (2012). https://doi.org/10.1016/0165-6074(89)90269-X
Zhao, H., Wang, Y., Duan, J., et al.: Multivariate time-series anomaly detection via graph attention network. In: ICDM (2020). https://doi.org/10.1109/ICDM50108.2020.00093
Xu, J., Wu, H., Wang, J., Long, M.: Anomaly transformer: time series anomaly detection with association discrepancy. In: ICLR (2021). arXiv preprint arXiv:2110.02642
Giuliano, C.: TranAD: deep Transformer networks for anomaly detection in multivariate time series data. In: Proceedings of the VLDB Endowment (2022). https://doi.org/10.48550/arXiv.2201.07284
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
He, H., Li, X., Chen, P., Chen, J., Song, W., Xi, Q. (2024). DGFormer: An Effective Dynamic Graph Transformer Based Anomaly Detection Model for IoT Time Series. In: Gao, H., Wang, X., Voros, N. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2023. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 562. Springer, Cham. https://doi.org/10.1007/978-3-031-54528-3_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-54528-3_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-54527-6
Online ISBN: 978-3-031-54528-3
eBook Packages: Computer ScienceComputer Science (R0)