Skip to main content
SpringerLink
Log in

Partitioning for Scalable Complex Event Processing on Data Streams

  • Conference paper
New Trends in Database and Information Systems II

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 312))

Abstract

Many applications processing dynamic data require to filter, aggregate, join as well as to recognize event patterns in streams of data in an online fashion. However, data analysis and complex event processing (CEP) on high volume and/or high rate streams are challenging tasks. Typically, partitioning techniques are leveraged for achieving low latency and scalable processing. Unfortunately, sequence-based operations such as CEP operations as well as long-running continuous queries make partitioning much more difficult than for batch-oriented approaches.

In this paper, we address this challenge by presenting partitioning strategies for CEP queries. We discuss two strategies for stream and pattern partitioning and we present a cost-based optimization approach for determining the number of partitions as well as the split points in the queries to achieve better load balancing and avoid congestions of processing nodes in a cluster environment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Brenna, L., Gehrke, J., Hong, M., Johansen, D.: Distributed event stream processing with non-deterministic finite automata. In: DEBS 2009, pp. 3:1–3:12 (2009)

    Google Scholar 

  2. Condie, T., Conway, N., Alvaro, P., Hellerstein, J.M., Elmeleegy, K., Sears, R.: MapReduce online. Technical Report UCB/EECS-2009-136, EECS Department, UC, Berkeley (October 2009)

    Google Scholar 

  3. Hirzel, M.: Partition and compose: Parallel complex event processing. In: DEBS Conference, pp. 191–200. ACM (2012)

    Google Scholar 

  4. Krämer, J., Seeger, B.: Semantics and implementation of continuous sliding window queries over data streams. ACM Trans. Database Syst. 34(1) (2009)

    Google Scholar 

  5. Neumeyer, L., Robbins, B., Nair, A., Kesari, A.: S4: Distributed stream computing platform. In: ICDMW 2010, pp. 170–177 (2010)

    Google Scholar 

  6. Schultz-Møller, N.P., Migliavacca, M., Pietzuch, P.: Distributed complex event processing with query rewriting. In: DEBS Conference, pp. 4:1–4:12. ACM (2009)

    Google Scholar 

  7. Viglas, S., Naughton, J.F.: Rate-based query optimization for streaming information sources. In: SIGMOD Conference, pp. 37–48 (2002)

    Google Scholar 

  8. Wu, E., Diao, Y., Rizvi, S.: High-performance complex event processing over streams. In: SIGMOD Conference, pp. 407–418 (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Automation, Technische Universität Ilmenau, Germany

    Omran Saleh, Heiko Betz & Kai-Uwe Sattler

Authors
  1. Omran Saleh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heiko Betz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kai-Uwe Sattler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Omran Saleh .

Editor information

Editors and Affiliations

  1. Aristotle University of Thessaloniki, Thessaloniki, Greece

    Nick Bassiliades

  2. Department of Mathematics and Informatics, Faculty of Sciences, University of Novi Sad, Novi Sad, Serbia

    Mirjana Ivanovic

  3. Faculty of Natural Science and Mathematics, Ss Cyril and Methodious University Institute for Informatics, Skopje, Macedonia

    Margita Kon-Popovska

  4. Department of Informatics, Aristotle University of Thessaloniki, University Campus, Thessaloniki, Greece

    Yannis Manolopoulos

  5. Paris Descartes University, Paris, France

    Themis Palpanas

  6. EECS Department, Northwestern University, Evanston, Illinois, USA

    Goce Trajcevski

  7. Department of Informatics, Aristotle University of Thessalonik, Thessaloniki, Greece

    Athena Vakali

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Saleh, O., Betz, H., Sattler, KU. (2015). Partitioning for Scalable Complex Event Processing on Data Streams. In: Bassiliades, N., et al. New Trends in Database and Information Systems II. Advances in Intelligent Systems and Computing, vol 312. Springer, Cham. https://doi.org/10.1007/978-3-319-10518-5_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-10518-5_15

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-10517-8

  • Online ISBN: 978-3-319-10518-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation