An Anomaly Detection Framework Based on Data Center Operation and Maintenance Data
Authors: 
Xin Liu, Jibin Wang, Shuwei QinAuthors Info & Claims
Xin Liu, Jibin Wang, Shuwei QinAuthors Info & ClaimsPages 883 - 889
Abstract
Data centers need to monitor various metrics of their different application platforms and applications in real-time. As the system architectures and application services of different application platforms within them become more complex, the requirements for their anomaly detection capabilities are higher. Therefore, this paper proposes an anomaly detection framework based on data center operation and maintenance data. The framework in this paper consists of three parts, including operation and maintenance data cleaning, data feature extraction, and model routing. It is used to select the appropriate model through model routing based on the indicators such as stability and periodicity obtained from data feature extraction of each application platform. At the same time, in order to enhance the expansion capability of the detection algorithm, a cloud-ground hybrid framework is used and a module for algorithm model management is designed to facilitate interaction with the cloud. After testing on SWAT and WADI datasets, the anomaly detection algorithm with the addition of model routing in the framework has good accuracy and recall performance compared to a single anomaly algorithm model, showing advantages in the task of identifying anomalies.
References
[1]
Su H, He Q, Guo B. KPI anomaly detection method for Data Center AIOps based on GRU-GAN[C]//2021 10th International Conference on Internet Computing for Science and Engineering. 2021: 23-29.
[2]
Chou J S, Telaga A S. Real-time detection of anomalous power consumption [J]. Renewable and Sustainable Energy Reviews, 2014, 33: 400-411.
[3]
Jaiswal A, Babu A R, Zadeh M Z, A survey on contrastive self-supervised learning [J]. Technologies, 2020, 9(1): 2.
[4]
Pang G, Shen C, Cao L, Deep learning for anomaly detection: A review [J]. ACM Computing Surveys (CSUR), 2021, 54 (2): 1-38.
[5]
Dang Y, Lin Q, Huang P. AIOps: real-world challenges and research innovations [C]//2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). IEEE, 2019: 4-5.
[6]
Prabhakar S P, Jololian L. Developing a machine learning course for anomaly detection [J]. Journal of Computing Sciences in Colleges, 2021, 37 (1): 84-91.
[7]
Chi L, Tian G, Mu Y, Fast non-local neural networks with spectral residual learning [C]//Proceedings of the 27th ACM International Conference on Multimedia. 2019: 2142-2151.
[8]
Hou X, Zhang L. Saliency detection: A spectral residual approach [C]//2007 IEEE Conference on computer vision and pattern recognition. Ieee, 2007: 1-8.
[9]
Ren H, Xu B, Wang Y, Time-series anomaly detection service at Microsoft [C]//Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 2019: 3009-3017.
[10]
Box G E P, Jenkins G M, Reinsel G C, Time series analysis: forecasting and control [M]. John Wiley & Sons, 2015.
[11]
Jap D, Stöttinger M, Bhasin S. Support vector regression: exploiting machine learning techniques for leakage modeling [C]//Proceedings of the Fourth Workshop on Hardware and Architectural Support for Security and Privacy. 2015: 1-8.
[12]
Standaert F X, Malkin T G, Yung M. A unified framework for the analysis of side-channel key recovery attacks [C]//Annual international conference on the theory and applications of cryptographic techniques. Springer, Berlin, Heidelberg, 2009: 443-461.
[13]
Taylor S J, Letham B. Forecasting at scale [J]. The American Statistician, 2018, 72 (1): 37-45.
[14]
Breiman L. Random forests [J]. Machine learning, 2001, 45 (1): 5-32.
[15]
Wang R, Nie K, Wang T, Deep learning for anomaly detection [C]//Proceedings of the 13th international conference on web search and data mining. 2020: 894-896.
[16]
Kingma D P, Welling M. Stochastic gradient VB and the variational auto-encoder [C]//Second International Conference on Learning Representations, ICLR. 2014, 19: 121.
[17]
Xu H, Chen W, Zhao N, Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications [C]//Proceedings of the 2018 world wide web conference. 2018: 187-196.
[18]
Zhao A, Qi L, Dong J, Dual channel LSTM based multi-feature extraction in gait for diagnosis of Neurodegenerative diseases [J]. Knowledge-Based Systems, 2018, 145: 91-97.
[19]
Kim T Y, Cho S B. Web traffic anomaly detection using C-LSTM neural networks [J]. Expert Systems with Applications, 2018, 106: 66-76.
[20]
Wu J, Yao L, Liu B, Multi-task learning based Encoder-Decoder: A comprehensive detection and diagnosis system for multi-sensor data [J]. Advances in Mechanical Engineering, 2021, 13 (5): 16878140211013138.
[21]
Chen L J, Ho Y H, Hsieh H H, ADF: An anomaly detection framework for large-scale PM2. 5 sensing systems [J]. IEEE Internet of Things Journal, 2017, 5 (2): 559-570.
[22]
Fuller W A. Introduction to statistical time series [M]. John Wiley & Sons, 2009.
[23]
Mathur A P, Tippenhauer N O. SWaT: A water treatment testbed for research and training on ICS security [C]//2016 international workshop on cyber-physical systems for smart water networks (CySWater). IEEE, 2016: 31-36.
[24]
Ahmed C M, Palleti V R, Mathur A P. WADI: a water distribution testbed for research in the design of secure cyber physical systems [C]//Proceedings of the 3rd international workshop on cyber-physical systems for smart water networks. 2017: 25-28.
Recommendations
An anomaly detection framework for time series data: An interval-based approach
AbstractDue to the high data volume and non-stationarity of time series data, it is very difficult to directly use the original data for anomaly detection. In this study, a novel framework of anomaly detection is proposed, whose intent is to ...
A Formal Framework for Program Anomaly Detection
RAID 2015: Proceedings of the 18th International Symposium on Research in Attacks, Intrusions, and Defenses - Volume 9404Program anomaly detection analyzes normal program behaviors and discovers aberrant executions caused by attacks, misconfigurations, program bugs, and unusual usage patterns. The merit of program anomaly detection is its independence from attack ...
Data streams anomaly detection algorithm based on self-set threshold
ICCIP '18: Proceedings of the 4th International Conference on Communication and Information ProcessingWith the rapid development of big data, smart city and artificial intelligence, anomaly detection technology for data streams has been widely used in big data detection, video surveillance, network intrusion detection, intelligent security analysis and ...
Comments
Information & Contributors
Information
Published In
December 2022
1396 pages
ISBN:9781450398343
DOI:10.1145/3584376
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 19 April 2023
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
RICAI 2022
RICAI 2022: 2022 4th International Conference on Robotics, Intelligent Control and Artificial Intelligence
December 16 - 18, 2022
Dongguan, China
Acceptance Rates
Overall Acceptance Rate 140 of 294 submissions, 48%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 58Total Downloads
- Downloads (Last 12 months)22
- Downloads (Last 6 weeks)7
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format