Approximating sliding windows by cyclic tree-like histograms for efficient range queries

Abstract

The issue of providing fast approximate answers to range queries on sliding windows with a small consumption of storage space is one of the main challenges in the context of data streams. On the one hand, the importance of this class of queries is widely accepted. They are indeed useful to compute aggregate information over the data stream, allowing us to extract from it more abstract knowledge than point queries. On the other hand, the usage of techniques like synopses based on histograms, sketches, sampling, and so on, makes effective those approaches which require multiple scans on data, which otherwise would be prohibitive from the computational point of view. Among the above techniques, histogram-based approaches are considered one of the most advantageous solutions, at least in case of range queries. It is a matter of fact that histograms show a very good capability of summarizing data preserving quick and accurate answers to range queries. In this paper, we propose a novel histogram-based technique to reduce sliding windows supporting approximate arbitrary range-sum queries. Our histogram, relying on a tree-based structure, is suitable to directly support hierarchical queries and, thus, drill-down and roll-up operations. In addition, the structure well supports sliding window shifting and quick query answering, since it operates in logarithmic time in the sliding window size. A bit-saving approach to encoding tree nodes allows us to compress the sliding window with a little price in terms of accuracy. The contribution of this work is thus not only the proposal of a new specific technique to tackle an important problem but also a deep analysis of the advantages given by the hierarchical approach combined with the bit-saving strategy. A careful experimental analysis validates the method showing its superiority w.r.t. the state of the art.

Introduction

Data streams are indefinite sequences of data which continuously vary in time, often very quickly. There are many application contexts characterized by the presence of data streams. Typically, their on-line analysis may offer knowledge useful for strategic analyses and statistics. This is the case of network monitoring, sensor networks, financial applications, security, telecommunication data management, Web applications, manufacturing, just to mention some examples. The data-stream analysis can be done by means of different types of queries. Queries can be distinguished into (1) point queries [28], (2) range queries [12], [50], [67], and (3) similarity queries [18]. Referring to a data stream of packets crossing a router, examples of the three query types are, respectively: (1) “Return the size of the k-th packet of the data stream”, (2) “Return the total amount of traffic crossing the router in a given time interval”, and (3) “Return true whether a pattern similar to a given dangerous pattern occurs in the data stream”.

Even though answering a query (of any type) requires to store the entire data stream, there are a large number of situations (even in the field of network analysis mentioned above) where the analysis of the most recent part of the data stream is enough to give meaningful information. From a semantic point of view, this means giving more importance to recent knowledge w.r.t. past one, assuming that recent information is more reliable and significant than older information [37]. Therefore, designing techniques able to process queries by considering a suitable portion of the data stream, called sliding window, has assumed particular importance in the recent literature [2], [7], [13], [23], [38], [60], [61], [64].

However, in order to give significance to the sliding window itself, its size (i.e., the number of most recent elements we keep in each instant) should be as large as possible. As a consequence any technique capable of compressing sliding windows yet maintaining a good approximate representation of the data distribution is certainly relevant in the field of data stream [6]. Observe that, reducing sliding windows allows us also to keep simultaneously more than just one approximate sliding window, in order to implement similarity queries like change mining queries [34], useful for trend analysis and, in general, to understand dynamics of the data stream itself. In sum, since in a typical streaming environment only limited memory resources are available [39], [59], reduction is a key factor allowing query processing also in case of multiple scans on data.

There are a number of properties that a sliding window reduction technique should satisfy. First, the reduced sliding window should maintain the semantic nature of the original data, in such a way that meaningful queries can be submitted to the reduced data in place of the original ones. Then, for a given kind of query, the accuracy of the reduced structure should be independent of the position where the query is applied in order to provide the user with an analysis tool supporting arbitrary queries. In addition, the reduction technique should support drill-down and roll-up operations.

Even though the general properties stated above are valid for each type of query, often the approximation techniques are designed for a specific kind of query for which they show a good behavior in terms of efficiency and precision. In this work we focus our attention on range queries. It is worth noting that this kind of query is particularly important from the application point of view. Consider for example an intrusion detection system whose sensors are located at choke points and capture all network traffic. The increase of the traffic coming from a range of source IPs or having a particular range of destination ports can be a sign of an attack. Moreover, similar statistics can be used by a network monitoring system to detect faults and congestion or to improve network load balancing.

In this paper, we propose a histogram-based technique for reducing sliding windows which supports arbitrary range queries and satisfies all the above properties. Our histogram, called c-Tree, differently from the traditional ones, is based on a hierarchical structure, in particular a tree. Its nodes contain, in an aggregation hierarchy, pre-computed range-sum queries, stored by a bit-saving encoding. For this reason, the structure directly supports the estimation of arbitrary range queries (in particular, range queries of type sum). Indeed, range queries are either embedded in the histogram or derivable by linear interpolation. Reduction derives from both the aggregation implemented by leaves of the tree (discretization), and the saving of bits obtained by representing range queries with less than 32 bits.

Our approach relies on a previous proposal presented in [14] and [16] for persistent data. However, histograms presented in the above papers are not applicable to data streams because they do not take into account the continuous updating of data. In contrast, the structure here proposed is efficiently dynamic, in the sense that each update can be executed in logarithmic time (w.r.t. the window size). In addition, answering a range query requires at most logarithmic time too. Observe that the hierarchical structure directly supports querying at different abstraction levels, thus allowing drill-down and roll-up operations. Finally, bucket summarization smoothes each data value by consulting the “neighborhood” values around it, working thus to remove the noise from data. But the main feature of our histogram concerns its high accuracy. This is particularly important since in order for the reduction technique to have a meaningful role in data analysis applications, the error should be either guaranteed or heuristically shown to be small (and this is our case).