LCAD: A Correlation Based Abnormal Pattern Detection Approach for Large Amount of Monitor Data

Abstract

The last decade has witnessed tremendous growths of Internet of Things(IoT). Numerous condition monitoring systems(CMS) are widely applied to monitor equipments simultaneously. With the help of CMS, a large variety of monitor data from a large number of equipments can be collected in a very short time. However, it is a non-trivial task to take full advantage of such large amounts of monitor data in the context of anomaly detection. In this paper, we propose an approach called Latent Correlation based Anomaly Detection(LCAD) that can quickly detect potential anomalies from a large amount of monitor data, which posits that abnormal ones are a small portion in a mass of similar individuals. Instead of focusing on each single monitor data series, we identify the abnormal pattern by modeling the latent correlation among multiple correlative monitor data series using the Latent Correlation Probabilistic Model(LCPM), a probabilistic distribution model which can help to detect anomalies depending on their relations with LCPM. In order to validate our approach, we conduct experiments on the real-world datasets and the experimental results show that when facing a large amount of correlative monitor data series LCAD has a better performance as compared to the previous anomaly detection approaches.