Alejandro Grez
Stijn Vansummeren
Skip Abstract Section
Abstract
Complex event recognition (CER) has emerged as the unifying field for technologies that require processing and correlating distributed data sources in real time. CER finds applications in diverse domains, which has resulted in a large number of proposals for expressing and processing complex events. Existing CER languages lack a clear semantics, however, which makes them hard to understand and generalize. Moreover, there are no general techniques for evaluating CER query languages with clear performance guarantees.
In this article, we embark on the task of giving a rigorous and efficient framework to CER. We propose a formal language for specifying complex events, called complex event logic (CEL), that contains the main features used in the literature and has a denotational and compositional semantics. We also formalize the so-called selection strategies, which had only been presented as by-design extensions to existing frameworks. We give insight into the language design trade-offs regarding the strict sequencing operators of CEL and selection strategies.
With a well-defined semantics at hand, we discuss how to efficiently process complex events by evaluating CEL formulas with unary filters. We start by introducing a formal computational model for CER, called complex event automata (CEA), and study how to compile CEL formulas with unary filters into CEA. Furthermore, we provide efficient algorithms for evaluating CEA over event streams using constant time per event followed by output-linear delay enumeration of the results.
- [1] . n.d. Apache FlinkCEP. Retrieved June 14, 2021 from https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/libs/cep/.Google Scholar
- [2] n.d. Esper Enterprise Edition Website. Retrieved January 28, 2021 from http://www.espertech.com/.Google Scholar
- [3] . 2003. Aurora: A data stream management system. In Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data.Google ScholarDigital Library
- [4] . 1995. Foundations of Databases: The Logical Level. Addison-Wesley.Google ScholarDigital Library
- [5] . 2004. Amit—The situation manager. VLDB Journal 13, 2 (2004), 177–203.Google ScholarDigital Library
- [6] . 2008. Efficient pattern matching over event streams. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. 147–160.Google ScholarDigital Library
- [7] . 1990. Algorithms for finding patterns in strings. In Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity (A). North Holland, 255–300.Google Scholar
- [8] . 1974. The Design and Analysis of Computer Algorithms. Pearson Education India.Google ScholarDigital Library
- [9] . 2008. Plan-based complex event detection across distributed sources. Proceedings of the VLDB Endowment 1, 1 (2008), 66–77.Google ScholarDigital Library
- [10] . 2017. Probabilistic complex event recognition: A survey. ACM Computing Surveys 50, 5 (2017), Article 71, 31 pages. https://doi.org/10.1145/3117809Google ScholarDigital Library
- [11] . 2013. Regular functions and cost register automata. In Proceedings of the 2013 28th Annual ACM/IEEE Symposium on Logic in Computer Science. 13–22. https://doi.org/10.1109/LICS.2013.65Google ScholarDigital Library
- [12] . 2020. Streamable regular transductions. Theoretical Computer Science 807 (2020), 15–41. https://doi.org/10.1016/j.tcs.2019.11.018Google ScholarDigital Library
- [13] . 2010. A rule-based language for complex event processing and reasoning. In Web Reasoning and Rule Systems. Lecture Notes in Computer Science, Vol. 6333. Springer, 42–57.Google ScholarCross Ref
- [14] . 2003. STREAM: The Stanford stream data manager (demonstration description). In Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data. 665.Google ScholarDigital Library
- [15] . 2006. The CQL continuous query language: Semantic foundations and query execution. VLDB Journal 15, 2 (2006), 121–142.Google ScholarDigital Library
- [16] . 2017. Complex event recognition languages: Tutorial. In Proceedings of the 11th ACM International Conference on Distributed and Event-Based Systems (DEBS’17). ACM, New York, NY, 7–10.Google ScholarDigital Library
- [17] . 2015. An event calculus for event recognition. IEEE Transactions on Knowledge and Data Engineering 27, 4 (2015), 895–908.Google ScholarDigital Library
- [18] . 2012. Logic-based event recognition. Knowledge Engineering Review 27, 4 (2012), 469–506.Google ScholarDigital Library
- [19] . 2007. Consistent streaming through time: A vision for event stream processing. In Proceedings of the 2007 Conference on Innovative Data Systems Research (CIDR’07). 363–374.Google Scholar
- [20] . 2013. Transductions and Context-Free Languages. Springer-Verlag.Google Scholar
- [21] . 2010. On notions of regularity for data languages. Theoretical Computer Science 411, 4 (2010), 702–715. https://doi.org/10.1016/j.tcs.2009.10.009Google ScholarDigital Library
- [22] . 2011. Two-variable logic on data words. ACM Transactions on Computational Logic 12, 4 (
July 2011), Article 27, 26 pages. https://doi.org/10.1145/1970398.1970403Google ScholarDigital Library - [23] . 2009. Complex event processing. IT–Information Technology 51, 5 (2009), 241–242.Google ScholarCross Ref
- [24] . 2003. A formal study of practical regular expressions. International Journal of Foundations of Computer Science 14, 6 (2003), 1007–1018.Google ScholarCross Ref
- [25] . 2009. On extended regular expressions. In Language and Automata Theory and Applications. Lecture Notes in Computer Science, Vol. 5457. Springer, 279–289.Google Scholar
- [26] . 2010. A resource-efficient event algebra. Science of Computer Programming 75, 12 (2010), 1215–1234.Google ScholarDigital Library
- [27] . 2000. NiagaraCQ: A scalable continuous query system for internet databases. In Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data. 379–390.Google ScholarDigital Library
- [28] . 2010. A logic-based, reactive calculus of events. Fundamenta Informaticae 105, 1–2 (2010), 135–161.Google ScholarDigital Library
- [29] . 2009. Raced: An adaptive middleware for complex event detection. In Proceedings of the 8th International Workshop on Adaptive and Reflective Middleware (ARM’09). Article 5, 6 pages.Google ScholarDigital Library
- [30] . 2010. TESLA: A formally defined event specification language. In Proceedings of the 4th ACM International Conference on Distributed Event-Based Systems (DEBS’10). 50–61.Google ScholarDigital Library
- [31] . 2012. Complex event processing with T-REX. Journal of Systems and Software 85, 8 (2012), 1709–1728.Google ScholarDigital Library
- [32] . 2012. Processing flows of information: From data stream to complex event processing. ACM Computing Surveys 44, 3 (2012), Article 15, 62 pages.Google ScholarDigital Library
- [33] . 2005. A General Algebra and Implementation for Monitoring Event Streams.
Technical Report . Cornell University.Google Scholar - [34] . 2006. Towards expressive publish/subscribe systems. In Proceedings of the 10th International Conference on Advances in Database Technology (EDBT’06). 627–644.Google Scholar
- [35] . 1989. Making data structures persistent. Journal of Computer and System Sciences 38, 1 (1989), 86–124.Google ScholarDigital Library
- [36] . 2007. First-order queries on structures of bounded degree are computable with constant delay. ACM Transactions on Computational Logic 8, 4 (2007), 1–18.Google ScholarDigital Library
- [37] . 2015. Document spanners: A formal approach to information extraction. Journal of the ACM 62, 2 (2015), Article 12, 51 pages. https://doi.org/10.1145/2699442Google ScholarDigital Library
- [38] . 2002. Two approaches to event definition. In Database and Expert Systems Applications. Lecture Notes in Computer Science, Vol. 2453. Springer, 547–556.Google ScholarCross Ref
- [39] . 2020. Complex event recognition in the big data era: A survey. VLDB Journal 29, 1 (2020), 313–352. https://doi.org/10.1007/s00778-019-00557-wGoogle ScholarCross Ref
- [40] . 2003. Issues in data stream management. ACM SIGMOD Record 32, 2 (2003), 5–14.Google ScholarDigital Library
- [41] . 2019. A formal framework for complex event processing. In Proceedings of the 22nd International Conference on Database Theory (IDCT’19). Article 5, 18 pages.Google Scholar
- [42] . 2020. On the expressiveness of languages for complex event recognition. In Proceedings of the 23rd International Conference on Database Theory (ICDT’20). Article 15, 17 pages. https://doi.org/10.4230/LIPIcs.ICDT.2020.15Google Scholar
- [43] . 2007. Automation, Production Systems, and Computer-Integrated Manufacturing. Prentice Hall.Google ScholarDigital Library
- [44] . 2008. On supporting Kleene closure over event streams. In Proceedings of the 2008 IEEE 24th International Conference on Data Engineering (ICDE’08). IEEE, Los Alamitos, CA, 1391–1393.Google ScholarDigital Library
- [45] . 2014. On load shedding in complex event processing. In Proceedings of the 17th International Conference on Database Theory (ICDT’14). 213–224.Google Scholar
- [46] . 2011. On the complexity of privacy-preserving complex event processing. In Proceedings of the 30th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS’11). 165–174.Google ScholarDigital Library
- [47] . 2013. Algorithmic techniques for processing data streams. Dagstuhl Follow-ups 5 (2013), 237–274.Google Scholar
- [48] . 1994. Finite-memory automata. Theoretical Computer Science 134, 2 (1994), 329–363. https://doi.org/10.1016/0304-3975(94)90242-9Google ScholarDigital Library
- [49] . 2013. Elements of Finite Model Theory. Springer Science & Business Media.Google Scholar
- [50] . 2011. E-Cube: Multi-dimensional event sequence analysis using hierarchical pattern query sharing. In Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data (SIGMOD’11). 889–900.Google ScholarDigital Library
- [51] . 1996. Rapide: A Language and Toolset for Simulation of Distributed Systems by Partial Orderings of Events. Technical Report. Computer Science Laboratory, Stanford University.Google Scholar
- [52] . 2017. StreamQRE: Modular specification and efficient evaluation of quantitative queries over streaming data. In Proceedings of the 38th ACM SIGPLAN Conference on Progamming Language Design and Implementation (PLDI’17). 693–708 https://doi.org/10.1145/3140587.3062369Google Scholar
- [53] . 2009. ZStream: A cost-based query processor for adaptively detecting composite events. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data (SIGMOD’09). ACM, New York, NY, 193–206.Google ScholarDigital Library
- [54] . 1994. Network intrusion detection. IEEE Network 8, 3 (1994), 26–41.Google ScholarDigital Library
- [55] . 2003. A framework for event composition in distributed systems. In Proceedings of the ACM/IFIP/USENIX 2003 International Conference on Middleware.Google ScholarDigital Library
- [56] . 2019. Composite event recognition for maritime monitoring. In Proceedings of the 13th ACM International Conference on Distributed and Event-Based Systems (DEBS’19). ACM, New York, NY, 163–174. https://doi.org/10.1145/3328905.3329762Google ScholarDigital Library
- [57] . 2008. Real time business intelligence in supply chain analytics. Information Management & Computer Security 16, 1 (2008), 28–48.Google ScholarCross Ref
- [58] . 2009. Distributed complex event processing with query rewriting. In Proceedings of the 3rd ACM International Conference on Distributed Event-Based Systems (DEBS’09). Article 4, 12 pages.Google ScholarDigital Library
- [59] . 2013. Enumerating with constant delay the answers to a query. In Proceedings of the 16th International Conference on Database Theory. 10–20.Google ScholarDigital Library
- [60] . 2013. Enumerating with constant delay the answers to a query. In Proceedings of the 16th International Conference on Database Theory (ICDT’13). 10–20.Google ScholarDigital Library
- [61] . 2013. Applications of symbolic finite automata. In Implementation and Application of Automata. Lecture Notes in Computer Science, Vol. 7982. Springer, 16–23.Google ScholarDigital Library
- [62] . 2007. What is “next” in event processing? In Proceedings of the 26th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS’07). 263–272.Google ScholarDigital Library
- [63] . 2006. High-performance complex event processing over streams. In Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data (SIGMOD’06). 407–418.Google ScholarDigital Library
- [64] . 2014. On complexity and optimization of expensive queries in complex event processing. In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data (SIGMOD’14). 217–228.Google ScholarDigital Library
- [65] . 1999. On the semantics of complex events in active database management systems. In Proceedings of the 15th International Conference on Data Engineering (ICDE’99).Google ScholarDigital Library
Index Terms
- A Formal Framework for Complex Event Recognition
Recommendations
Complex event forecasting with prediction suffix trees
AbstractComplex event recognition (CER) systems have become popular in the past two decades due to their ability to “instantly” detect patterns on real-time streams of events. However, there is a lack of methods for forecasting when a pattern might occur ...
Towards complex actions for complex event processing
DEBS '13: Proceedings of the 7th ACM international conference on Distributed event-based systemsComplex actions are a natural extension for complex event processing languages needed by many applications like emergency management. In particular interactions with external actuators that are common in those applications pose challenges that need to ...
Complex event recognition using constrained low-rank representation
Complex event recognition is the problem of recognizing events in long and unconstrained videos. In this extremely challenging task, concepts have recently shown a promising direction where core low-level events (referred to as concepts) are annotated ...
Comments