Skip to main content
SpringerLink
Log in

Query Optimization in Heterogenous Event Processing Federations

  • SCHWERPUNKTBEITRAG
  • Published:
Datenbank-Spektrum Aims and scope Submit manuscript

Abstract

Continuous processing of event streams evolved to an important class of data management over the last years and will become even more important due to novel applications such as the Internet of Things. Because systems for data stream and event processing have been developed independent of each other, often in competition and without the existence of any standards, the Stream Processing System (SPS) landscape is extremely heterogeneous today. To overcome the problems caused by this heterogeneity, a novel event processing middleware, the Java Event Processing Connectivity (JEPC), has been presented recently. However, despite the fact that SPSs can be accessed uniformly using JEPC, their different performance profiles caused by different algorithms and implementations remain. This gives the opportunity to query optimization, because individual system strengths can be exploited. In this paper, we present a novel query optimizer that exploits the technical heterogeneity in a federation of different unified SPSs. Taking into account different performance profiles of SPSs, we address query plan partitioning, candidate selection, and reducing inter-system communication in order to improve the overall query performance. We suggest a heuristic that finds a good initial mapping of sub-plans to a set of heterogenous SPSs. An experimental evaluation clearly shows that heterogeneous federations outperform homogeneous federations, in general, and that our heuristic performs well in practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Listing 1
Abb. 1
Abb. 2
Abb. 3
Abb. 4
Abb. 5

Similar content being viewed by others

Notes

  1. We anonymized the actual system names due to legal reasons.

Literatur

  1. Abadi D et al (2005) The design of the Borealis stream processing engine. CIDR 277–289

  2. Baumgärtner L et al (2015) Complex event processing for reactive security monitoring in virtualized computer systems. DEBS 22–33

  3. Botan I et al (2010) A demonstration of the MaxStream federated stream processing system. ICDE 1093–1096

  4. Dindar N, Tatbul N, Miller R, Haas L, Botan I (2013) Modeling the execution semantics of stream processing engines with SECRET. VLDB J 22(4):421–446

  5. Gulisano V, Jimenez-Peris R, Patino-Martinez M, Soriente C, Valduriez P (2012) StreamCloud: an elastic and scalable data streaming system. TPDS 23(12):2351–2365

  6. Hoßbach B (2015) Design and implementation of a middleware for uniform, federated and dynamic event processing. PhD thesis, University of Marburg

  7. Hoßbach B, Seeger B (2013) Anomaly management using complex event processing. EDBT 149–154

  8. Hoßbach B, Freisleben B, Seeger B (2012) Reaktives cloud monitoring mit complex event processing. Datenbank Spektrum 12(1):33–42

  9. Hoßbach B, Glombiewski N, Morgen A, Ritter F, Seeger B (2013) JEPC: the java event processing connectivity. Datenbank Spektrum 13(3):167–178

  10. Jain N et al (2008) Towards a streaming SQL standard. PVLDB 1(2):1379–1390

  11. Krämer J, Seeger S (2009) Semantics and implementation of continuous sliding window queries over data streams. ACM Trans Database Syst 34(1): 4:1–4:49

  12. Lim H, Han Y, Babu S (2013) How to fit when no one size fits. CIDR

  13. Opher E (2010) Event processing: past, present and future. PVLDB 3(1–2):1651–1652

  14. Park Y, King R, Nathan S, Most W, Andrade H (2011) Evaluation of a high-volume, low-latency market data processing system implemented with IBM middleware. Softw Pract Exper 42(1):37–56

  15. Patroumpas K, Sellis T (2012) Event processing and real-time monitoring over streaming traffic data. W2GIS 116–133

  16. Pinnecke M (2015) Konzept und prototypische Implementierung eines föderativen Complex Event Processing Systems mit Operatorverteilung. BTW Workshops 233–242

  17. Sheth A, Larson J (1990) Federated database systems for managing distributed, heterogeneous, and autonomous databases. CSUR 22(3):183–236

  18. Tatbul N (2010) Streaming data integration: challenges and opportunities. ICDEW 155–158

  19. Wu K et al (2007) Challenges and experience in prototyping a multi-modal stream analytic and monitoring application on system S. VLDB 1185–1196

Download references

Acknowledgement

This work was supported by the German Federal Ministry of Education and Research (Bundesministerium für Bildung und Forschung, BMBF) under grant no. 16BY1206A.

Author information

Authors and Affiliations

  1. Otto-von-Guericke University Magdeburg, 39106, Magdeburg, Germany

    Marcus Pinnecke

  2. Database Research Group, University of Marburg, 35032, Marburg, Germany

    Bastian Hoßbach

Authors
  1. Marcus Pinnecke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bastian Hoßbach
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcus Pinnecke.

Additional information

This is an extended version of the paper “Konzept und prototypische Implementierung eines föderativen Complex Event Processing Systems mit Operatorverteilung” [16] selected for the special DASP issue Best Workshop Papers of BTW 2015.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pinnecke, M., Hoßbach, B. Query Optimization in Heterogenous Event Processing Federations. Datenbank Spektrum 15, 193–202 (2015). https://doi.org/10.1007/s13222-015-0195-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13222-015-0195-0

Keywords

Search

Navigation