Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Business Intelligence for the Real-Time Enterprise

International Workshop on Business Intelligence for the Real-Time Enterprise

BIRTE 2014, BIRTE 2013: Enabling Real-Time Business Intelligence pp 133–149Cite as

Efficient Pattern Detection Over a Distributed Framework

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 206))

Abstract

In recent past, work has been done to parallelize pattern detection queries over event stream, by partitioning the event stream on certain keys or attributes. In such partitioning schemes the degree of parallelization totally relies on the available partition keys. A limited number of partitioning keys, or unavailability of such partitioning attributes noticeably affect the distribution of data among multiple nodes, and is a reason of potential data skew and improper resource utilization. Moreover, majority of the past implementations of complex event detection are based on a single machine, hence, they are immune to potential data skew that could be seen in a real distributed environment. In this study, we propose an event stream partitioning scheme that without considering any key attributes partitions the stream over time-windows. This scheme efficiently distributes the event stream partitions across network, and detects pattern sequences in distributed fashion. Our scheme also provides an effective means to minimize potential data skew and handles a substantial number of pattern queries across network.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   34.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   44.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Diao, Y., Immerman, N., Gyllstrom, D.: Sase+: An Agile Language for Kleene Closure Over Event Streams. ACM Press, New York (2007)

    Google Scholar 

  2. Agrawal, J., Diao, Y., Gyllstrom, D., Immerman, N.: Efficient pattern matching over event streams. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 147–160. ACM (2008)

    Google Scholar 

  3. Mei, Y., Madden, S.: Zstream: a cost-based query processor for adaptively detecting composite events. In: Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data, pp. 193–206. ACM (2009)

    Google Scholar 

  4. Wu, E., Diao, Y., Rizvi, S.: High-performance complex event processing over streams. In: Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data, pp. 407–418. ACM (2006)

    Google Scholar 

  5. Liu, M., Ray, M., Rundensteiner, E.A., Dougherty, D.J., Gupta, C., Wang, S., Ari, I., Mehta, A.: Processing nested complex sequence pattern queries over event streams. In: Proceedings of the Seventh International Workshop on Data Management for Sensor Networks, pp. 14–19. ACM (2010)

    Google Scholar 

  6. Ramakrishnan, R., Cheng, M., Livny, M., Seshadri, P.: What’s next? sequence queries. In: Proceedings of International Conferene Management of Data. Citeseer (1994)

    Google Scholar 

  7. Liu, M., Li, M., Golovnya, D., Rundensteiner, E.A., Claypool, K.: Sequence pattern query processing over out-of-order event streams. In: IEEE 25th International Conference on Data Engineering, ICDE 2009, pp. 784–795. IEEE (2009)

    Google Scholar 

  8. Law, Y.N., Wang, H., Zaniolo, C.: Query languages and data models for database sequences and data streams. In: Proceedings of the Thirtieth International Conference on Very Large Data Bases, vol. 30, pp. 492–503. VLDB Endowment (2004)

    Google Scholar 

  9. Seshadri, P., Livny, M., Ramakrishnan, R.: Sequence query processing. ACM SIGMOD Rec. 23, 430–441 (1994). ACM

    Article  Google Scholar 

  10. Zuo, X., Zhou, Y., Zhao, C.-H.: Elastic non-contiguous sequence pattern detection for data stream monitoring. In: Yin, H., Tino, P., Corchado, E., Byrne, W., Yao, X. (eds.) IDEAL 2007. LNCS, vol. 4881, pp. 599–608. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  11. Gao, C., Wei, J., Xu, C., Cheung, S.: Sequential event pattern based context-aware adaptation. In: Proceedings of the Second Asia-Pacific Symposium on Internetware, p. 3. ACM (2010)

    Google Scholar 

  12. Gyllstrom, D., Agrawal, J., Diao, Y., Immerman, N.: On supporting kleene closure over event streams. In: ICDE, vol. 8, pp. 1391–1393 (2008)

    Google Scholar 

  13. Demers, A.J., Gehrke, J., Panda, B., Riedewald, M., Sharma, V., White, W.M., et al.: Cayuga: a general purpose event monitoring system. In: CIDR, vol. 7, pp. 412–422 (2007)

    Google Scholar 

  14. Balkesen, C., Dindar, N., Wetter, M., Tatbul, N.: Rip: run-based intra-query parallelism for scalable complex event processing. In: Proceedings of the 7th ACM International Conference on Distributed Event-Based Systems, pp. 3–14. ACM (2013)

    Google Scholar 

  15. Brenna, L., Gehrke, J., Hong, M., Johansen, D.: Distributed event stream processing with non-deterministic finite automata. In: Proceedings of the Third ACM International Conference on Distributed Event-Based Systems, p. 3. ACM (2009)

    Google Scholar 

  16. Chakravarthy, S., Krishnaprasad, V., Anwar, E., Kim, S.K.: Composite events for active databases: semantics, contexts and detection. VLDB 94, 606–617 (1994)

    Google Scholar 

  17. Peng, S., Li, Z., Li, Q., Chen, Q., Pan, W., Liu, H., Nie, Y.: Event detection over live and archived streams. In: Wang, H., Li, S., Oyama, S., Hu, X., Qian, T. (eds.) WAIM 2011. LNCS, vol. 6897, pp. 566–577. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  18. Zdonik, S., Sibley, P., Rasin, A., Sweetser, V., Montgomery, P., Turner, J., Wicks, J., Zgolinski, A., Snyder, D., Humphrey, M., Williamson, C.: Streaming for dummies (2004)

    Google Scholar 

  19. Hirzel, M.: Partition and compose: parallel complex event processing. In: Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems, pp. 191–200. ACM (2012)

    Google Scholar 

  20. Schultz-Møller, N.P., Migliavacca, M., Pietzuch, P.: Distributed complex event processing with query rewriting. In: Proceedings of the Third ACM International Conference on Distributed Event-Based Systems, p. 4. ACM (2009)

    Google Scholar 

  21. Sadoghi, M., Singh, H., Jacobsen, H.A.: Towards highly parallel event processing through reconfigurable hardware. In: Proceedings of the Seventh International Workshop on Data Management on New Hardware, pp. 27–32. ACM (2011)

    Google Scholar 

  22. Hirzel, M., Andrade, H., Gedik, B., Kumar, V., Losa, G., Nasgaard, M., Soule, R., Wu, K.: Spl stream processing language specification. IBM Research, Yorktown Heights, NY, USA, Technical report RC24 897 (2009)

    Google Scholar 

  23. Stonebraker, M., Çetintemel, U., Zdonik, S.: The 8 requirements of real-time stream processing. ACM SIGMOD Rec. 34(4), 42–47 (2005)

    Article  Google Scholar 

  24. Golab, L., Özsu, M.T.: Issues in data stream management. ACM SIGMOD Rec. 32(2), 5–14 (2003)

    Article  Google Scholar 

  25. Wang, Y., Cao, K., Zhang, X.: Complex event processing over distributed probabilistic event streams. Comput. Math. Appl. 66(10), 1808–1821 (2013)

    Article  Google Scholar 

  26. Mani, M.: Efficient event stream processing: handling ambiguous events and patterns with negation. In: Xu, J., Yu, G., Zhou, S., Unland, R. (eds.) DASFAA Workshops 2011. LNCS, vol. 6637, pp. 415–426. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  27. Kawashima, H., Kitagawa, H., Li, X.: Complex event processing over uncertain data streams. In: 2010 International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC), pp. 521–526. IEEE (2010)

    Google Scholar 

  28. Jiang, Q., Chakravarthy, S.: Scheduling strategies for a data stream management system. Computer Science & Engineering, BNCOD, pp. 16–30 (2004)

    Google Scholar 

  29. Sharaf, M.A., Labrinidis, A., Chrysanthis, P.K.: Scheduling continuous queries in data stream management systems. Proc. VLDB Endow. 1(2), 1526–1527 (2008)

    Article  Google Scholar 

  30. Wu, J., Tan, K.-L., Zhou, Y.: QoS-oriented multi-query scheduling over data streams. In: Zhou, X., Yokota, H., Deng, K., Liu, Q. (eds.) DASFAA 2009. LNCS, vol. 5463, pp. 215–229. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  31. Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and issues in data stream systems. In: Proceedings of the Twenty-First ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, pp. 1–16. ACM (2002)

    Google Scholar 

  32. Babcock, B., Babu, S., Datar, M., Motwani, R., Thomas, D.: Operator scheduling in data stream systems. VLDB J. Int. J. Very Large Data Bases 13(4), 333–353 (2004)

    Article  Google Scholar 

  33. Babcock, B., Babu, S., Motwani, R., Datar, M.: Chain: operator scheduling for memory minimization in data stream systems. In: Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data, pp. 253–264. ACM (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Mathematics and Computer Science (IMADA), University of Southern Denmark, Odense, Denmark

    Ahmed Khan Leghari, Martin Wolf & Yongluan Zhou

Authors
  1. Ahmed Khan Leghari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yongluan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmed Khan Leghari .

Editor information

Editors and Affiliations

  1. Hewlett-Packard, Palo Alto, USA

    Malu Castellanos

  2. Hitachi Laboratories, Santa Clara, California, USA

    Umeshwar Dayal

  3. Aalborg University, Aalborg, Denmark

    Torben Bach Pedersen

  4. Intel Labs and MIT, Cambridge, Massachusetts, USA

    Nesime Tatbul

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Leghari, A.K., Wolf, M., Zhou, Y. (2015). Efficient Pattern Detection Over a Distributed Framework. In: Castellanos, M., Dayal, U., Pedersen, T., Tatbul, N. (eds) Enabling Real-Time Business Intelligence. BIRTE BIRTE 2014 2013. Lecture Notes in Business Information Processing, vol 206. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-46839-5_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-46839-5_9

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-46838-8

  • Online ISBN: 978-3-662-46839-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation