Self-adaptive statistical process control for anomaly detection in time series

doi:10.1016/j.eswa.2016.03.029

Expert Systems with Applications

Volume 57, 15 September 2016, Pages 324-336

https://doi.org/10.1016/j.eswa.2016.03.029 Get rights and content

Highlights

•
We model anomaly detection as a statistical testing based on fuzzy set theory.
•
Detection rate and false alarm rate almost are not affected by different K.
•
K optimization is necessary for AUC performance improvement.
•
Fuzzification can effectively reduce false alarm rate.
•
This approach results in high AUC performance and reduces the detection time.

Abstract

Anomaly detection in time series has become a widespread problem in the areas such as intrusion detection and industrial process monitoring. Major challenges in anomaly detection systems include unknown data distribution, control limit determination, multiple parameters, training data and fuzziness of ‘anomaly’. Motivated by these considerations, a novel model is developed, whose salient feature is a synergistic combination of statistical and fuzzy set-based techniques. We view anomaly detection problem as a certain statistical hypothesis testing. Meanwhile, ‘anomaly’ itself includes fuzziness, therefore, can be described with fuzzy sets, which bring a facet of robustness to the overall scheme. Intensive fuzzification is engaged and plays an important role in the successive step of hypothesis testing. Because of intensive fuzzification, the proposed algorithm is distribution-free and self-adaptive, which solves the limitation of control limit and multiple parameters. The framework is realized in an unsupervised mode, leading to great portability and scalability. The performance is assessed in terms of ROC curve on university of California Riverside repository. A series of experiments show that the proposed approach can significantly increase the AUC, while the false alarm rate is improved. In particular, it is capable of detecting anomalies at the earliest possible time.

Introduction

Anomaly detection in time series provides significant information for numerous applications. For example, it can be used to detect intrusions in network data (Abadeh, Mohamadi, & Habibi, 2011), fraud detection (Ahmed, Mahmood, & Islam, 2016), incident faults in industrial process (Brighenti & Sanz-Bobi, 2011). Anomalies in time series can manifest in terms of the changes in the amplitude of data, or can be associated with the changes in the shape of temporal waveforms. In light of this, we categorize anomalies into two types: anomalies in amplitude and anomalies in shape. For example, it is an anomaly in amplitude that is a premature ventricular contraction in electrocardiogram (ECG) signals in Fig. 1 and it is an anomaly in shape that is a premature Poppet withdrawal in a Space Shuttle Marotta Valve time series shown in Fig. 2. These anomalous parts are highlighted in red in both figures.

Anomalies are time series that are the least similar to all other time series and depart from the bounds of the state of statistical control which exists when certain critical process variables remain close to their target values and do not change perceptibly. Time series that stay in a state of statistical control are called in-control data (normal data), otherwise, are called out-of-control data (anomaly). In statistical process control, control charts are used to determine if a process is in a state of statistical control. As shown in Fig. 3, a control chart consists of:

(1)
Points representing a statistic of measurements of a quality characteristic in samples taken from the process at different times or different data.
(2)
The mean of this statistic using all the samples at which a center line is drawn.
(3)
Upper control limits (UCL) and lower control limits (LCL) that indicate the threshold at which the process output is considered statistically ‘unlikely’.

Anomaly detection in time series is more challenging due to several reasons. First, it makes control limits very important decision aids. Control limits provide information about the process behavior and have no intrinsic relationship to any specification targets. In practice, the process mean (the center line) may not coincide with the specified value of the quality characteristic, because the process design simply cannot deliver the process characteristic at the desired level. It is also a key challenge to select a threshold instead of process mean. Second, anomaly is a more complex concept. For example, if one sample’s characteristic is equal to UCL - ε (ε is an infinitesimal positive number), it is normal. But if one sample’s characteristic is equal to UCL + ε, it becomes difficult to determine whether it is normal or abnormal. Third, many other algorithms require several parameters whose values are to be determined. This requires to acquire large amounts of training data, therefore, most of algorithms are realized in the supervised mode.

Due to these major challenges including unknown data distribution, control limit determination, multiple parameters, training data and fuzziness of ‘anomaly’ in anomaly detection systems, a synergistic combination of statistical and fuzzy set-based technique is proposed in this paper. We view anomaly detection as a statistical hypothesis testing and introduce a definition based on control chart in statistical process control. Because the process mean may not coincide with the specified value of the quality, we do not adopt the mean of samples’ characteristic, but a threshold. Anomaly could be a more complex concept, so the threshold should be fuzzy. Fuzzy set theory is taken into account to provide a better characterization of the boundary between normal and abnormal. What’s more, the inequality (>, ≤) in statistical hypothesis test is treated as a fuzzy predicate (the degree of inclusion). Intensive fuzzification process is adopted to realize related parameters determination which is self-adaptive. Therefore, the values of parameters are not required to be specified by the user. Due to the use of fuzzy set theory, statistical hypothesis testing in this paper is a distribution-free and totally unsupervised model. What’s more, the overall scheme is self-adaptive. The utility is demonstrated using synthetic and real data sets. We have conducted a number of studies that show the effectiveness of our algorithm to detect anomalies in time series data.

The paper is structured as follows. Section 2 reviews some previous works on anomaly detection. Section 3 illustrates how anomaly detection can be viewed as a statistical hypothesis testing. A fuzzy-statistical algorithm for detecting anomalies is described in Section 4. We present some applications and perform extensive evaluation in Section 5 to demonstrate both the utility and ability to detect anomalies. Finally, the paper is summarized and concluded in Section 6.

Section snippets

Related works

The broad categories of anomaly detection techniques are: classification-based techniques (Koc , Mazzuchi , Sarkani , 2012, Dangelo , Palmieri , Ficco , Rampone , 2015), nearest neighbor-based techniques (Ceclio , Ottewill , Pretlove , Thornhill , 2014, Sajjad , Bouk , Yousaf , 2015, Lin , Ke , Tsai , 2015), clustering-based techniques (Ahmed , Mahmood , Maher , 2015, Lee , Kim , Kim , 2011), statistical techniques (Zhang , Lu , Zhang , Ruan , 2016, Pierazzi , Casolari , Colajanni , Marchetti ,

Anomaly detection based on statistical process control

According to statistical process control, the statistic of a quality characteristic should be defined, which represents the anomaly score of the data. When the anomaly score is larger, the data is more likely to be anomaly. So, we just consider how large the anomaly score is and do not need to care how small it is. That is to say, we also need to determine the upper control limit, with regard to lower control limit. Now we illustrate how any anomaly detection scheme can be viewed as a

Self-adaptive detection model

As mentioned earlier, ‘anomaly’ is a complex concept and therefore the threshold determination can be realized by engaging fuzzy sets. In this section, we show how to determine threshold using fuzzy set theory and a certain fuzzification process that leads to the treatment of inequalities (>, ≤) as fuzzy predicates. The aim of fuzzification is to achieve optimized results. Let us also note that because of fuzzification, the algorithm is a distribution-free statistical testing.

Experiments and discussions

We begin experiments by showing the usefulness of the proposed algorithm for synthetic data and real-life data, including anomalies in shape and amplitude. Then, we perform several experiments to evaluate its performance. Finally, we contrast our algorithm against several baseline algorithms to show that the proposed algorithm is able to efficiently find anomalies. In our experiments, it is workable without training process. K can assume any value.

Conclusion

In this work, we have mainly developed a self-adaptive algorithm for finding anomalies in time series. A key feature of the algorithm is a synergistic combination of both statistical and fuzzy set-based theories. Exploiting fuzzy set theory in statistical process control, the detection is a distribution-free and unsupervised model. K optimization is necessary for AUC improvement. In this case, detection rate and false alarm rate almost are not affected by different K. False alarm rate has been

Acknowledgments

The work of this paper is funded by the project of National Natural Science Foundation of China (No.91520204) and the project of National High Technology Research and Development Program of China (863 Program) (No. 2015AA015405).

References (39)

AbadehM.S. et al.
Design and analysis of genetic fuzzy systems for intrusion detection in computer networks
Expert Systems with Applications
(2011)
AhmedM. et al.
A survey of anomaly detection techniques in financial domain
Future Generation Computer Systems
(2016)
AlbertettiF. et al.
Change points detection in crime-related time series: an on-line fuzzy approach based on a shape space representation
Applied Soft Computing
(2016)
AppiceA. et al.
Dealing with temporal and spatial correlations to classify outliers in geophysical data streams
Information Sciences
(2014)
AshfaqA.B. et al.
Information theoretic feature space slicing for statistical anomaly detection
Journal of Network and Computer Applications
(2014)
BondavalliA. et al.
Differential analysis of operating system indicators for anomaly detection in dependable systems: an experimental study
Measurement
(2016)
DangeloG. et al.
An uncertainty-managing batch relevance-based approach to network anomaly detection
Applied Soft Computing
(2015)
DerracJ. et al.
Fuzzy nearest neighbor algorithms: taxonomy, experimental analysis and prospects
Information Sciences
(2014)
DrugmanT.
Using mutual information in supervised temporal event detection: application to cough detection
Biomedical Signal Processing and Control
(2014)
HaJ. et al.
A precise ranking method for outlier detection
Information Sciences
(2015)

HarrouF. et al.

Improved principal component analysis for anomaly detection: application to an emergency department

Computers $&$ Industrial Engineering

(2015)

HuangY.P. et al.

Hybrid intelligent methods for arrhythmia detection and geriatric depression diagnosis

Applied Soft Computing

(2014)

KadriF. et al.

Seasonal ARMA-based spc charts for anomaly detection: application to emergency department systems

Neurocomputing

(2016)

KocL. et al.

A network intrusion detection system based on a hidden naïve bayes multiclass classifier

Expert Systems with Applications

(2012)

KoulaouzidisG. et al.

Prompt and accurate diagnosis of ventricular arrhythmias with a novel index based on phase space reconstruction of ECG

International Journal of Cardiology

(2015)

KumarageH. et al.

Distributed anomaly detection for industrial wireless sensor networks based on fuzzy data modelling

Journal of Parallel and Distributed Computing

(2013)

LeeJ. et al.

Integrating independent component analysis and local outlier factor for plant-wide process monitoring

Journal of Process Control

(2011)

LeeS. et al.

Self-adaptive and dynamic clustering for online anomaly detection

Expert Systems with Applications

(2011)

LemosA. et al.

Adaptive fault detection and diagnosis using an evolving fuzzy classifier

Information Sciences

(2013)

Cited by (31)

Distributed SFA-CA monitoring approach for nonstationary plant-wide process and its application on a vinyl acetate monomer process
2022, Process Safety and Environmental Protection
Citation Excerpt :
Owing to the increasing demand in plant safety and product quality, process monitoring and fault diagnosis play an increasingly important role. With the application of distributed control system in modern process industry and the development of computing technology, large amounts of process data are restored, leading the development of data-driven methods (Amin et al., 2021; Arunthavanathan et al., 2021; Dominic et al., 2015; Nan et al., 2007; Zheng et al., 2016). Multivariate statistical process monitoring (MSPM) methods, such as principal component analysis (PCA) and partial least square (PLS), can extract the main features from high-dimensional data.
Driven by the nonstationary and large-scale characteristics in the modern plant-wide processes, this paper proposes a weighting matrix decomposition based distributed slow feature analysis (SFA) task. First, the long-term equilibrium relationships between nonstationary variables are explored by CA to transform the nonstationary variables to be stationary. Based on this, SFA monitoring model is built for whole stationary space to extract the dynamic features. Considering the similar features in the monitoring model, the weighting matrix of SFA is decomposed into subspaces to strengthen the interpretation of local process information. Finally, Bayesian Inference (BI) is used to incorporate the results of each subspace to generate a global monitoring statistic. The effectiveness of the proposed method is tested on the Vinyl Acetate Monomer process.
Object oriented time series exploration: Applied to power consumption analysis of embedded systems
2021, Expert Systems with Applications
Citation Excerpt :
Finally, the contribution is discussed and summarized in Sections 7 and 8, respectively. Anomalies in time series (Zheng, Li, & Zhao, 2016) usually relate to the amplitude, shape, or time features of temporal waveforms. They can be defined as anomaly scores, e.g., minimal, maximal (extreme) values, some statistical features, etc.
Performance monitoring and anomaly detection are major issues in designing and maintaining electronic devices and systems. In recent years, they become more difficult due to the increased complexity of hardware and software. Hence, an important point is to collect representative signal samples and reveal characteristic features allowing to evaluate device operational profiles. This results in the need of an efficient time series analysis. This problem is considered in relevance to embedded systems and Internet of Things devices. The paper presents a new scheme of decomposing time series by introducing higher level objects targeted at the searched system properties. They create a compact state model which facilitates deriving knowledge on system behaviour to validate correctness of its operation. The collected samples are aggregated into objects according to predefined similarity metrics, these objects can be traced, correlated, and merged with relevant operational log events. For this purpose, a set of original algorithms have been composed and included in the developed software tool. The presented approach has been evaluated on a representative dataset obtained from commercial Holter devices and was used to explore their energy consumption efficiency.
Efficient on-line anomaly detection for ship systems in operation
2019, Expert Systems with Applications
Citation Excerpt :
An extensive number of anomaly detection methods are described in the literature and used extensively in a wide variety of applications in various industries. The available techniques comprise (Chandola et al., 2009; Kanarachos, Christopoulos, Chroneos, & Fitzpatrick, 2017; Olson, Judd, & Nichols, 2018; Zheng, Li, & Zhao, 2016): classification methods that are rule-based, or based on Neural Networks, Bayesian Networks or Support Vector Machines; nearest neighbour based methods, including k nearest neighbour and relative density; clustering based methods; and statistical and fuzzy set-based techniques, including parametric and non-parametric methods based on histograms or kernel functions. The fundamental approaches to the problem of anomaly detection can be divided into three categories (Chandola et al., 2009; Hodge & Austin, 2004):
We propose novel modifications to an anomaly detection methodology based on multivariate signal reconstruction followed by residuals analysis. The reconstructions are made using Auto Associative Kernel Regression (AAKR), where the query observations are compared to historical observations called memory vectors, representing normal operation. When the data set with historical observations grows large, the naive approach where all observations are used as memory vectors will lead to unacceptable large computational loads, hence a reduced set of memory vectors should be intelligently selected. The residuals between the observed and the reconstructed signals are analysed using standard Sequential Probability Ratio Tests (SPRT), where appropriate alarms are raised based on the sequential behaviour of the residuals.
The modifications we introduce include: a novel cluster based method to select memory vectors to be considered by the AAKR, which gives an extensive reduction in computation time; a generalization of the distance measure, which makes it possible to distinguish between explanatory and response variables; and a regional credibility estimation used in the residuals analysis, to let the time used to identify if a sequence of query vectors represents an anomalous state or not, depend on the amount of data situated close to or surrounding the query vector.
We demonstrate how the anomaly detection method and the proposed modifications can be successfully applied for anomaly detection on a set of imbalanced benchmark data sets, as well as on recent data from a marine diesel engine in operation.
Evaluating the benefits of using proactive transformed-domain-based techniques in fraud detection tasks
2019, Future Generation Computer Systems
Citation Excerpt :
This happens when it is important to characterize the involved elements on the basis of the time factor [22]. The information extracted from the time series can be exploited in order to perform different tasks, such as those related to the risk analysis (e.g., Credit Scoring [23] and Stock Forecasting [24]) and Information Security (e.g., Fraud Detection [25] and Intrusion Detection [26]) ones. In other words, the relationship between time series and our fraud detection approach must be sought in the analysis, performed in the frequency domain, of patterns given by the feature values of a transaction.
The exponential growth in the number of E-commerce transactions indicates a radical change in the way people buy and sell goods and services, a new opportunity offered by a huge global market, where they may choose sellers or buyers on the basis of multiple criteria (e.g., economic, logistical, ethical, sustainability, etc.), without being forced to use the traditional brick-and-mortar criterion. If, on the one hand, such a scenario offers an enormous control to people, both at private and corporate level, allowing them to filter their needs by adopting a large range of criteria, on the other hand, it has contributed to the growth of fraud cases related to the involved electronic instruments of payment, such as credit cards. The Big Data Information Security for Sustainability is a research branch aimed to face these issues in relation to the potential implications in the field of sustainability, proposing effective solutions to design safe environments in which the people can operate and by exploiting the benefits related to new technologies. The fraud detection systems are a significant example of such solutions, although the techniques adopted by them are typically based on retroactive strategies, which are incapable of preventing fraudulent events. In this perspective, this paper aims to investigate the benefits related to the adoption of proactive fraud detection strategies, instead of the canonical retroactive ones, theorizing those solutions that can lead toward practical effective implementations. We evaluate two previously experimented novel proactive strategies, one based on the Fourier transform, and one based on the Wavelet transform, which are used in order to move the data (i.e., financial transactions) into a new domain, where they are analyzed and an evaluation model is defined. Such strategies allow a fraud detection system to operate by using a proactive approach, since they do not exploit previous fraudulent transactions, overcoming some important problems that reduce the effectiveness of the canonical retroactive state-of-the-art solutions. Potential benefits and limitations of the proposed proactive approach have been evaluated in a real-world credit card fraud detection scenario, by comparing its performance to that of one of the most used and performing retroactive state-of-the-art approaches (i.e. Random Forests).
Fuzzified Cuckoo based Clustering Technique for Network Anomaly Detection
2018, Computers and Electrical Engineering
Citation Excerpt :
Results are reported by applying these metrics to aforementioned datasets. Table 11 shows the corresponding results using K-means, Decision Tree, PSO, CSO (MSE), CSO (MSE, SI), FCOAC [22], SADA [23], SSAD [24], TVCPSO [25] and F-CBCT. Further, it can be noticed from Fig. 5 that TVCPSO gives comparable performance in most of the cases but FPR of the proposed F-CBCT is quite less (in the considered datasets) as compared to the proposed one.
With the increasing penetration of security threats, the severity of their impact on the underlying network has increased manifold. Hence, a robust anomaly detection technique, Fuzzified Cuckoo based Clustering Technique (F-CBCT), is proposed in this paper which operates in two phases: training and detection. The training phase is supported using Decision Tree followed by an algorithm based on hybridization of Cuckoo Search Optimization and K-means clustering. In the designed algorithm, a multi-objective function based on Mean Square Error and Silhouette Index is employed to evaluate the two simultaneous distance functions namely-Classification measure and Anomaly detection measure. Once the system is trained, detection phase is initiated in which a fuzzy decisive approach is used to detect anomalies on the basis of input data and distance functions computed in the previous phase. Experimental results in terms of detection rate (96.86%), false positive rate (1.297%), accuracy (97.77%) and F-Measure (98.30%) prove the effectiveness of the proposed model.
Multivariate time series anomaly detection: A framework of Hidden Markov Models
2017, Applied Soft Computing Journal
Citation Excerpt :
The first difficulty arises because of the lack of a concise and operational anomaly definition [12]. Unusual points (exhibiting too high or too low values) and unexpected subsequences (e.g., shape changes) [13] appearing in univariate time series can be considered as anomaly. Unlike these definitions, multivariate techniques do not only deal with the abnormal values or subsequences in each time series but also investigate the relationships among these variables.
In this study, we develop an approach to multivariate time series anomaly detection focused on the transformation of multivariate time series to univariate time series. Several transformation techniques involving Fuzzy C-Means (FCM) clustering and fuzzy integral are studied. In the sequel, a Hidden Markov Model (HMM), one of the commonly encountered statistical methods, is engaged here to detect anomalies in multivariate time series. We construct HMM-based anomaly detectors and in this context compare several transformation methods. A suite of experimental studies along with some comparative analysis is reported.

View all citing articles on Scopus

View full text

Self-adaptive statistical process control for anomaly detection in time series

Highlights

Abstract

Introduction

Section snippets

Related works

Anomaly detection based on statistical process control

Self-adaptive detection model

Experiments and discussions

Conclusion

Acknowledgments

Expert Systems with Applications

Future Generation Computer Systems

Applied Soft Computing

Information Sciences

Journal of Network and Computer Applications

Measurement

Applied Soft Computing

Information Sciences

Biomedical Signal Processing and Control

Information Sciences

Computers & Industrial Engineering

Applied Soft Computing

Neurocomputing

Expert Systems with Applications

International Journal of Cardiology

Journal of Parallel and Distributed Computing

Journal of Process Control

Expert Systems with Applications

Information Sciences

Computers $&$ Industrial Engineering