Radhya Sahal
ABSTRACT
With the ever increasing number of IoT devices getting connected, an enormous amount of streaming data is being produced with very high velocity. In order to process these large number of data streams, a variety of stream processing platforms and query engines are emerging. In the stream query processing, an infinite data stream is divided into small chunks of finite data using a window operator. Window size and its type play an important role in the performance of any stream query engine. Due to the dynamic nature of IoT, data stream rate fluctuates very often, thus impeding the performance of query engines. In this work, we investigated the impact of any changes in data stream rates over the performance of a distributed query engine (e.g. Flink - https://flink.apache.org/). Our evaluation results indicate a direct impact of any changes in stream rate and window size over the performance of the engines. We propose an adaptive and dynamic query window size and type selector to improve the resilience of query processing engines. We consider several characteristics of input data streams, application workload, and resource constraints and proposes an optimal stream query window size and type for stream query execution.
- João Gama and Mohamed Medhat Gaber. 2007. Learning from data streams: processing techniques in sensor networks. Springer. Google Scholar
Digital Library
- Christoph Hochreiner, Michael Vögler, Stefan Schulte, and Schahram Dustdar. 2016. Elastic stream processing for the internet of things. In Cloud Computing (CLOUD), 2016 IEEE 9th International Conference on. IEEE, 100--107.Google ScholarCross Ref
- On Evaluating the Impact of Changes in IoT Data Streams Rate over Query Window Configurations
Recommendations
Technology of Continuous Query Optimization over Data Streams
ISISE '08: Proceedings of the 2008 International Symposium on Information Science and Engieering - Volume 02The characteristic of data stream is that it has a huge size and its data change continually, which need to be responded quickly, since the times of query is limited. The continuous query and data stream approximate query model are introduced in this ...
Window query processing for joining data streams with relations
CASCON '07: Proceedings of the 2007 conference of the center for advanced studies on Collaborative researchQuery processing for data streams raises challenges that cannot be directly handled by existing database management systems (DBMS). Most related work in the literature mainly focuses on developing techniques for a dedicated data stream management system ...
Exploiting predicate-window semantics over data streams
The continuous sliding-window query model is used widely in data stream management systems where the focus of a continuous query is limited to a set of the most recent tuples. In this paper, we show that an interesting and important class of queries ...
Comments