Sequential Anomaly Detection Using Feedback and Prioritized Experience Replay

Abstract

Time series anomaly detection is essential because it helps in identifying faulty sensors and malicious behaviour in real-time. Most of the research work on anomaly detection revolves around density-based unsupervised learning techniques for batch data and forecasting (threshold-based) techniques for streaming data. Typically in streaming data, we continuously encounter concept drifts due to which the forecasting approaches’ threshold becomes insignificant with time. Also, forecasting techniques cannot identify sequential anomalies, as they try to forecast as per the ingested data. The reason behind less implementations using supervised learning for anomaly detection is because of the class imbalance problem in the dataset and unavailability of the labels. Most of the anomaly datasets contain 5% outliers due to which any learning model will overfit on normal data class and will not be able to learn about the anomalous class. In this work, we address these issues using Prioritized Experience Replay and introduce a novel state function which incorporates feedback to identify sequential anomalies. We evaluate our model on the Yahoo benchmark dataset, which contains 367 time-series datasets (each testing different aspects of anomaly detection), four smart home energy datasets and Numenta Anomaly benchmark datasets consisting of 58 time series data. The paper exhibits better performance of the proposed approach over the baseline approaches across different anomaly datasets.