Abstract
Modern database systems are in need of supporting highly scalable transactions of data updates and efficient queries over data simultaneously for the real-time applications. One solution to reach the demand is to implement query optimization techniques on the online transaction processing (OLTP) systems. The materialized view is considered as a panacea to improve query latency. However, it also involves a significant cost of maintenance which trades away transaction performance. In this paper, we develop materialized views on a distributed log-structured merge-tree (LSM-tree), which is a well-known structure adopted to improve data write performance. We examine the design space and conclude several design features for the implementation of view on LSM-tree. An asynchronous approach with two optimizations are proposed to decouple the view maintenance with transaction process. Under the asynchronous update, we also provide consistency query for views. Experiments on TPC-H benchmark show our method achieves better performance than straightforward methods on different workloads.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Delete as well as update, since the record with index will not be actually deleted but just marked for deletion.
- 2.
This is the most commonly used view schema for projection.
References
Alibaba: OceanBase. https://github.com/alibaba/oceanbase/
Couchdb. https://couchdb.apache.org/
Record locking. https://en.wikipedia.org/wiki/Record_locking
Segment Tree. https://en.wikipedia.org/wiki/Segment_tree
TPC-H. http://www.tpc.org/tpch/
Agrawal, P., Silberstein, A., Cooper, B.F., et al. Asynchronous view maintenance for VLSD databases. In: SIGMOD, pp. 179–192. ACM (2009)
Bello, R.G., Dias, K., et al.: Materialized views in Oracle. VLDB, pp. 659–664. Morgan Kaufmann Publishers Inc., San Francisco (1998)
Berenson, H., Bernstein, P., Gray, J., Melton, J., O’Neil, E., O’Neil, P.: A critique of ANSI SQL isolation levels. In: Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, SIGMOD 1995, pp. 1–10. ACM, New York (1995)
Chang, F., Dean, J., Ghemawat, S., et al.: Bigtable: a distributed storage system for structured data. ACM Trans. Comput. Syst. 26(2), 4 (2008)
Ghemawat, S., Gobioff, H., Leung, S.-T.: The Google file system. In: SOSP, vol. 37, pp. 29–43 (2003)
Goldstein, J., Larson, P.-Å.: Optimizing queries using materialized views: a practical, scalable solution. In: ACM SIGMOD Record, vol. 30, pp. 331–342. ACM (2001)
Katsis, Y., Ong, K.W., Papakonstantinou, Y., et al.: Utilizing IDs to accelerate incremental view maintenance. In: SIGMOD, pp. 1985–2000. ACM (2015)
Lakshman, A., Malik, P.: Cassandra: a decentralized structured storage system. Oper. Syst. Rev. 44(2), 35–40 (2010)
Levandoski, J.J., Lomet, D.B., Sengupta, S.: The BW-Tree: a B-tree for new hardware platforms. In: ICDE, pp. 302–313 (2013)
LevelDB: LevelDB. http://leveldb.org/
Zaharioudakis, M., Cochrane, R., et al.: Answering complex SQL queries using automatic summary tables. ACM SIGMOD Rec. 29, 105–116 (2000)
O’Neil, P., Cheng, E., Gawlick, D., O’Neil, E.: The log-structured merge-tree (LSM-tree). Acta Inf. 33(4), 351–385 (1996)
Sears, R., Ramakrishnan, R.: BLSM: a general purpose log structured merge tree. In: SIGMOD, pp. 217–228. ACM (2012)
Tan, W., Tata, S., Tang, Y., Fong, L.L.: Diff-index: differentiated index in distributed log-structured data stores. In: EDBT, pp. 700–711 (2014)
Zhou, J., Larson, P.-A., Elmongui, H.G.: Lazy maintenance of materialized views. pp. 231–242. VLDB Endowment (2007)
Acknowledgements
This work is supported by National Science Foundation of China under grant numbers 61702189, 61432006 and 61672232, and Youth Science and Technology - “Yang Fan” Program(17YF1427800). Huiqi Hu is the corresponding author.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Duan, H., Hu, H., Qian, W., Ma, H., Wang, X., Zhou, A. (2018). Incremental Materialized View Maintenance on Distributed Log-Structured Merge-Tree. In: Pei, J., Manolopoulos, Y., Sadiq, S., Li, J. (eds) Database Systems for Advanced Applications. DASFAA 2018. Lecture Notes in Computer Science(), vol 10828. Springer, Cham. https://doi.org/10.1007/978-3-319-91458-9_42
Download citation
DOI: https://doi.org/10.1007/978-3-319-91458-9_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91457-2
Online ISBN: 978-3-319-91458-9
eBook Packages: Computer ScienceComputer Science (R0)