Abstract
Stream processing has become an important research issue with the increase in stream data sources. Many stream processing systems need to reference non-streaming resources such as database relations to answer real world queries. Since database relation is a shared entity, it may be updated during the continuous query (CQ) execution by other database clients resulting in inconsistent query results (partly using the relation before update and partly after update). For this problem, an isolation model is needed to define the way in which these updates are reflected in the output of the stream-relation join. In this work we propose an incremental CQ processing approach with isolation guarantees which makes use of a monitor operator to transform the relational updates into stream tuples. Since database relations tend to be large, an in-memory T*-Tree index is used to increase the stream-relation join efficiency. Experiments are performed to prove that guaranteeing isolation solves the inconsistency problem and to show that the incremental computation and indexing improves the query throughput significantly.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
The term mixed join is used by Neil Conway in [10] for the stream-relation join.
- 2.
The Rstream operator is commonly used to stream the mixed-join results.
- 3.
For simplicity, we assume that a new tuple arrives at every time instant t.
- 4.
For the sake of experiments, the inconsistent tuples are checked manually.
References
Arasu, A., Babu, S., Widom, J.: The CQL continuous query language: semantics foundations and query execution. VLDB J. 15(2), 121–142 (2006)
Abadi, D.J., Carney, D., Cetintemel, U., Cherniack, M., Convey, C., Lee, S., Stonebraker, M., Tatbul, N., Zdonik, S.: Aurora: a new model and architecture for data stream management. VLDB J. 12(2), 120–139 (2003)
Storm project. https://storm.apache.org/. Accessed 16 Feb 2016
Arasu, A., Babcock, B., Babu, S., Cieslewicz, J., Datar, M., Ito, K., Motwani, R., Srivastava, U., Widom, J.: STREAM: the stanford data stream management system. Technical report, Stanford InfoLab (2003). IEEE Data Eng. Bull. 26(1)
Abadi, D.J., Ahmad, Y., Balazinska, M., Cherniack, M., Hyon Hwang, J., Lindner, W., Maskey, A.S., Rasin, E., Ryvkina, E., Tatbul, N., Xing, Y., Zdonik, S.: The design of the borealis stream processing engine. In: Proceedings of the CIDR, pp. 277–289 (2005)
Wu, Y., Tan, K.: ChronoStream: elastic stateful stream computation in the cloud. In: Proceedings of the ICDE, pp. 723–734 (2015)
Cetintemel, U., Du, J., Kraska, T., Madden, S., Maier, D., Meehan, J., Pavlo, A., Stonebraker, M., Sutherland, E., Tatbul, N., Tufte, K., Wang, H., Zdonik, S.B.: S-Store: a streaming NewSQL system for big velocity applications. In: Proceedings of VLDB, pp. 1633–1636 (2014)
Chandramouli, B., Goldstein, J., Barnett, M., DeLine, R., Fisher, D., Platt, J.C., Terwilliger, J.F., Wernsing, J.: Trill: a high-performance incremental query processor for diverse analytics. In: Proceedings of VLDB, pp. 401–412 (2014)
Shaikh, S.A., Watanabe, Y., Wang, Y., Kitagawa, H.: Smart query execution for event-driven stream processing. In: Proceedings of IEEE BigMM (2016, to appear)
Conway, N.: Transactions and data stream processing, pp. 1–28 (2008)
Oyamada, M., Kawashima, H., Kitagawa, H.: Continuous query processing with concurrency control: reading updatable resources consistently. In: Proceedings of ACM SAC, pp. 788–794 (2013)
Lehman, T.J., Carey, M.J.: A study of index structures for main memory database management systems. In: Proceedings of VLDB, pp. 294–303 (1986)
Choi, K.-R., Kim, K.-C.: T*-tree: a main memory database index structure for real time applications. In: Proceedings of International Workshop on Real-Time Computing Systems and Applications, pp. 81–88 (1996)
Golab, L., Tamer Özsu, M.: Update-pattern-aware modeling and processing of continuous queries. In: Proceedings of ACM SIGMOD, pp. 658–669 (2005)
Botan, I., Fischer, P.M., Kossmann, D., Tatbul, N.: Transactional stream processing. In: Proceedings of EDBT, pp. 204–215 (2012)
Meehan, J., Tatbul, N., Zdonik, S., Aslantas, C., Cetintemel, U., Jiang, D., Kraska, T., Madden, S., Maier, D., Pavlo, A., Stonebraker, M., Tufte, K., Wang, H.: S-Store: streaming meets transaction processing. Proc. VLDB 8(13), 2134–2145 (2015)
Kallman, R., Kimura, H., Natkins, J., Pavlo, A., Rasin, A., Zdonik, S., Jones, E.P.C., Madden, S., Stonebraker, M., Zhang, Y., Hugg, J., Abadi, D.J.: H-Store: a high-performance, distributed main memory transaction processing system. Proc. VLDB 1(2), 1496–1499 (2008)
Weikum, G., Vossen, G.: Transactional Information Systems: Theory, Algorithms, and the Practice of Concurrency Control and Recovery. Morgan Kaufmann Publishers, San Francisco (2001)
Chakraborty, A., Singh, A.: A partition-based approach to support streaming updates over persistent data in an active DW. In: Proceedings of IPDPS, pp. 1–11 (2009)
Chandramouli, B., Goldstein, J., Duan, S.: Temporal analytics on big data for web advertising. In: Proceedings of ICDE, pp. 90–101 (2012)
Golab, L., Johnson, T.: Consistency in a stream warehouse. In: Proceedings of CIDR, pp. 114–122 (2011)
Golab, L., Johnson, T., Seidel, J.S., Shkapenyuk, V.: Stream warehousing with DataDepot. In: Proceedings of ACM SIGMOD, pp. 847–854 (2009)
Jubatus. http://jubat.us/. Accessed 16 Feb 2016
Naeem, M.A., Dobbie, G., Weber, G., Alam, S.: R-MESHJOIN for near-real-time data warehousing. In: Proceedings of International Workshop on DOLAP, pp. 53–60 (2010)
Polyzotis, N., Skiadopoulos, S., Vassiliadis, P., Simitsis, A., Frantzell, N.E.: Supporting streaming updates in an active DW. In: Proceedings of ICDE, pp. 476–485 (2007)
Polyzotis, N., Skiadopoulos, S., Vassiliadis, P., Simitsis, A., Frantzell, N.: Meshing streaming updates with persistent data in an active data warehouse. IEEE TKDE 20(7), 976–991 (2008)
Aho, A., Hopcroft, J., Ullman, J.D.: The Design and Analysis of Computer Algorithms. Addison-Wesley Publishing Company, Boston (1974)
Comer, D.: Ubiquitous B-tree. ACM Comp. Surv. 11(2), 121–137 (1979)
OCCI. http://www.oracle.com/. Accessed 18 Dec 2015
Acknowledgements
This research was partly supported by the program “Research and Development on Real World Big Data Integration and Analysis” of MEXT, Japan.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Shaikh, S.A., Chao, D., Nishimura, K., Kitagawa, H. (2016). Incremental Continuous Query Processing over Streams and Relations with Isolation Guarantees. In: Hartmann, S., Ma, H. (eds) Database and Expert Systems Applications. DEXA 2016. Lecture Notes in Computer Science(), vol 9827. Springer, Cham. https://doi.org/10.1007/978-3-319-44403-1_20
Download citation
DOI: https://doi.org/10.1007/978-3-319-44403-1_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-44402-4
Online ISBN: 978-3-319-44403-1
eBook Packages: Computer ScienceComputer Science (R0)