Towards Hybrid Online On-Demand Querying of Realtime Data with Stateful Complex Event Processing
Emerging Big Data applications in areas like e-commerce and energy industry require both online and on-demand queries to be performed over vast and fast data arriving as streams. These present novel challenges to Big Data management systems. Complex Event Processing (CEP) is recognized as a high performance online query scheme which in particular deals with the velocity aspect of the 3-V’s of Big Data. However, traditional CEP systems do not consider data variety and lack the capability to embed ad hoc queries over the volume of data streams. In this paper, we propose H2O, a stateful complex event processing framework, to support hybrid online and on-demand queries over realtime data. We propose a semantically enriched event and query model to address data variety. A formal query algebra is developed to precisely capture the stateful and containment semantics of online and on-demand queries. We describe techniques to achieve the interactive query processing over realtime data featured by efficient online querying, dynamic stream data persistence and on-demand access. The system architecture is presented and the current implementation status reported.
- Research Organization:
- City of Los Angeles Department
- Sponsoring Organization:
- USDOE Office of Electricity (OE)
- DOE Contract Number:
- OE0000192
- OSTI ID:
- 1332549
- Report Number(s):
- DOE-USC-00192-97
- Resource Relation:
- Conference: IEEE International Conference on Big Data - Workshop on Big Data and Smarter Cities Santa Clara, CA, USA 9-Oct-13
- Country of Publication:
- United States
- Language:
- English
Similar Records
An Informatics Approach to Demand Response Optimization in Smart Grids
Challenge Online Time Series Clustering For Demand Response A Theory to Break the ‘Curse of Dimensionality'