Abstract
Today’s in-memory column stores make heavy use of bit-packed data structures in order to reduce the required amount of main-memory and to improve the performance of memory-bound algorithms by trading more CPU cycles for less data that needs to be transferred over the memory-bus.
In this paper, we propose vertical bit-packing as a slightly modified alternative compared to classic bit-packing approaches, compressing an array of integer values with a known and finite value set so that each value is stored using the minimal required amount of bits. Vertical bit-packing aims to fully exploit the data parallelism provided by the existing on-chip vector processing units of modern x86-64 CPUs as they provide speedup potentials at no additional hardware cost.
In particular, we propose Vertical Bit-Packing and Aligned Vertical Bit-Packing as an alternative to the classic approach called Horizontal Bit-Packing. We show that the proposed techniques can save between one and two instructions per decompressed value block, outperforming the classic approach in some bit-cases with up to 12 %.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abel, J., Balasubramanian, K., Bargeron, M., Craver, T., Phlipot, M.: Applications tuning for streaming SIMD extensions. Intel Technol. J. Q2, 1–13 (1999)
Browne, S., Dongarra, J., Garner, N., Ho, G., Mucci, P.: A portable programming interface for performance evaluation on modern processors. Int. J. High Perform. Comput. Appl. 14, 189–204 (2000)
Chhugani, J., Nguyen, A.D., Lee, V.W., Macy, W., Hagog, M., Chen, Y.-K., Baransi, A., Kumar, S., Dubey, P.: Efficient implementation of sorting on multi-core SIMD CPU architecture. In: VLDB (2008)
Färber, F., Cha, S.K., Primsch, J., Bornhövd, C., Sigg, S., Lehner, W.: SAP HANA database: data management for modern business applications. In: SIGMOD (2012)
Grund, M., Krüger, J., Plattner, H., Zeier, A., Cudre-Mauroux, P., Madden, S.: HYRISE: a main memory hybrid storage engine. In: VLDB (2010)
Krüger, J., Kim, C., Grund, M., Satish, N., Schwalb, D., Chhugani, J., Plattner, H., Dubey, P., Zeier, A.: Fast updates on read-optimized databases using multi-core CPUs. In: VLDB (2011)
Lemire, D., Boytsov, L.: Decoding billions of integers per second through vectorization. In: CoRR (2012)
Lemke, C., Sattler, K.-U., Faerber, F., Zeier, A.: Speeding up queries in column stores. In: Bach Pedersen, T., Mohania, M.K., Tjoa, A.M. (eds.) DAWAK 2010. LNCS, vol. 6263, pp. 117–129. Springer, Heidelberg (2010)
Li, Y., Patel, J.M.: BitWeaving: fast scans for main memory data processing. In: SIGMOD (2013)
Manegold, S., Boncz, P.A., Kersten, M.L.: Optimizing database architecture for the new bottleneck: memory access. In: VLDB (2000)
Plattner, H., Zeier, A.: In-Memory Data Management: An Inflection Point for Enterprise Applications. Springer, New York (2011)
Schlegel, B., Gemulla, R., Lehner, W.: Fast integer compression using SIMD instructions. In: Proceedings of the Sixth International Workshop on Data Management on New Hardware, pp. 34–40. ACM (2010)
Willhalm, T., Oukid, I., Mueller, I., Faerber, F.: Vectorizing database column scans with complex predicates. In: AMDS (2013)
Willhalm, T., Popovici, N., Boshmaf, Y., Plattner, H., Zeier, A., Schaffner, J.: SIMD-scan: ultra fast in-memory table scan using on-chip vector processing units. Proc. VLDB Endow. 2(1), 385–394 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Faust, M., Grund, M., Berning, T., Schwalb, D., Plattner, H. (2014). Vertical Bit-Packing: Optimizing Operations on Bit-Packed Vectors Leveraging SIMD Instructions. In: Han, WS., Lee, M., Muliantara, A., Sanjaya, N., Thalheim, B., Zhou, S. (eds) Database Systems for Advanced Applications. DASFAA 2014. Lecture Notes in Computer Science(), vol 8505. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-43984-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-662-43984-5_10
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-43983-8
Online ISBN: 978-3-662-43984-5
eBook Packages: Computer ScienceComputer Science (R0)