Multiple Continuous Outlier Detection over Data Stream

Abstract

This paper studies the problem of Multiple Continuous Outlier Detection (MCOD for short) over data stream, a fundamental problem in the domain of streaming data management. Let $\mathcal {S}$ be the set of streaming data, and $\mathcal {Q}$ be the outlier detection query (query for short) workload. It contains a set of queries with different query parameters. Each query q(n, s, k, r) within $\mathcal {Q}$ monitors objects in $\mathcal {S}$ that are generated within the last q(n) time units. Whenever q(s) time units pass, q will return outliers within the range threshold r that do not satisfy k neighbor thresholds to the system. Some efforts are proposed to support MCOD, but they incur highly running cost both in computational and space, which cannot efficiently work under data stream.

In this paper, we propose a novel framework named Maximal Common Neighbour (MCN for short) over data stream. It is based on the following observation. That is, if query parameters under two queries q and $q'$ are similar, neighbours of an object o under q are likely to be neighbours of o under $q'$. Accordingly, we propose a novel index named Common Neighbour Tree (CN-Tree for short) to maintain neighbours of objects under different queries. It organizes neighbours of each object based on similar relationships among queries, so as to avoid redundant neighbour maintenance. In addition, we propose a group of efficient algorithms to support CN-Tree maintenance. Finally, we conduct extensive performance studies on large real and synthetic data sets, which demonstrate that our new framework could efficiently support MCOD over data stream.