Abstract
Compression in column-oriented databases has been proven to offer both performance enhancements and reductions in storage consumption. This is especially true for read access as compressed data can directly be processed for query execution.Nevertheless, compression happens to be disadvantageous when it comes to write access due to unavoidable re-compression: write-access requires significantly more data to be read than involved in the particular operation, more tuples may have to be modified depending on the compression algorithm, and table-level locks have to be acquired instead of row-level locks as long as no second version of the data is stored. As an effect the duration of a single modification — both insert and update — limits both throughput and response time significantly. In this paper, we propose to use an additional write-optimized buffer to maintain the delta that in conjunction with the compressed main store represents the current state of the data. This buffer facilitates an uncompressed, column-oriented data structure. To address the mentioned disadvantages of data compression, we trade write-performance for query-performance and memory consumption by using the buffer as an intermediate storage for several modifications which are then populated as a bulk in a merge operation. Hereby, the overhead created by one single re-compression is shared among all recent modifications. We evaluated our implementation inside SAP’s in memory column store. We then analyze the different parameters influencing the merge process, and make a complexity analysis. Finally, we show optimizations regarding resource consumption and merge duration.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abadi, D., Madden, S., Ferreira, M.: Integrating compression and execution in column-oriented database systems. In: SIGMOD 2006, pp. 671–682. ACM, New York (2006)
Abadi, D.J.: Query Execution in Column-Oriented Database Systems. PhD thesis
Boncz, P.A., Manegold, S., Kersten, M.L.: Database architecture optimized for the new bottleneck: Memory access. In: VLDB, pp. 54–65 (1999)
Boncz, P.A., Zukowski, M., Nes, N.: Monetdb/x100: Hyper-pipelining query execution. In: CIDR, pp. 225–237 (2005)
Brown, K.P., Mehta, M., Carey, M.J., Livny, M.: Towards automated performance tuning for complex workloads. In: VLDB, pp. 72–84 (1994)
Chen, Z., Gehrke, J., Korn, F.: Query optimization in compressed database systems. In: SIGMOD 2001, pp. 271–282. ACM, New York (2001)
Copeland, G.P., Khoshafian, S.: A decomposition storage model. In: SIGMOD Conference, pp. 268–279 (1985)
French, C.D.: “one size fits all” database architectures do not work for DDS. In: SIGMOD Conference, pp. 449–450 (1995)
French, C.D.: Teaching an OLTP database kernel advanced data warehousing techniques. In: ICDE, pp. 194–198 (1997)
Harizopoulos, S., Liang, V., Abadi, D.J., Madden, S.: Performance tradeoffs in read-optimized databases. In: VLDB, pp. 487–498 (2006)
Holloway, A.L., DeWitt, D.J.: Read-optimized databases, in depth. PVLDB 1(1), 502–513 (2008)
Legler, T., Lehner, W., Ross, A.: Data mining with the SAP NetWeaver BI accelerator. In: VLDB 2006, pp. 1059–1068. VLDB Endowment (2006)
Pang, H., Carey, M.J., Livny, M.: Multiclass query scheduling in real-time database systems. IEEE Trans. Knowl. Data Eng. 7(4), 533–551 (1995)
Plattner, H.: A common database approach for OLTP and OLAP using an in-memory column database. In: SIGMOD Conference, pp. 1–2 (2009)
Ramamurthy, R., DeWitt, D.J., Su, Q.: A case for fractured mirrors. In: VLDB, pp. 430–441 (2002)
Rao, J., Ross, K.A.: Making b\(^{\mbox{+}}\)-trees cache conscious in main memory. In: SIGMOD Conference, pp. 475–486 (2000)
Rappaport, R.L.: File structure design to facilitate on-line instantaneous updating. In: SIGMOD 1975, pp. 1–14. ACM, New York (1975)
Severance, D.G., Lohman, G.M.: Differential files: their application to the maintenance of large databases. ACM Trans. Database Syst. 1(3), 256–267 (1976)
Stonebraker, M., Abadi, D.J., Batkin, A., Chen, X., Cherniack, M., Ferreira, M., Lau, E., Lin, A., Madden, S., O’Neil, E.J., O’Neil, P.E., Rasin, A., Tran, N., Zdonik, S.B.: C-store: A column-oriented dbms. In: VLDB, pp. 553–564 (2005)
Westmann, T., Kossmann, D., Helmer, S., Moerkotte, G.: The implementation and performance of compressed databases. SIGMOD Rec. 29(3), 55–67 (2000)
Willhalm, T., Popovici, N., Boshmaf, Y., Plattner, H., Zeier, A., Schaffner, J.: Simd-scan: Ultra fast in-memory table scan using on-chip vector processing units. PVLDB 2(1), 385–394 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Krueger, J., Grund, M., Tinnefeld, C., Plattner, H., Zeier, A., Faerber, F. (2010). Optimizing Write Performance for Read Optimized Databases. In: Kitagawa, H., Ishikawa, Y., Li, Q., Watanabe, C. (eds) Database Systems for Advanced Applications. DASFAA 2010. Lecture Notes in Computer Science, vol 5982. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12098-5_23
Download citation
DOI: https://doi.org/10.1007/978-3-642-12098-5_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12097-8
Online ISBN: 978-3-642-12098-5
eBook Packages: Computer ScienceComputer Science (R0)