research-article

RAPID: In-Memory Analytical Query Processing Engine with Extreme Performance per Watt

Authors:

Cagri Balkesen,

Georgios Giannikis,

Sandeep Agrawal,

Venkatanathan Varadarajan,

Anand Viswanathan,

Balakrishnan Chandrasekaran,

Eric SedlarAuthors Info & Claims

SIGMOD '18: Proceedings of the 2018 International Conference on Management of Data

Pages 1407 - 1419

https://doi.org/10.1145/3183713.3190655

Published: 27 May 2018 Publication History

Abstract

Today, an ever increasing amount of transistors are packed into processor designs with extra features to support a broad range of applications. As a consequence, processors are becoming more and more complex and power hungry. At the same time, they only sustain an average performance for a wide variety of applications while not providing the best performance for specific applications. In this paper, we demonstrate through a carefully designed modern data processing system called RAPID and a simple, low-power processor specially tailored for data processing that at least an order of magnitude performance/power improvement in SQL processing can be achieved over a modern system running on today's complex processors. RAPID is designed from the ground up with hardware/software co-design in mind to provide architecture-conscious extreme performance while consuming less power in comparison to the modern database systems. The paper presents in detail the design and implementation of RAPID, a relational, columnar, in-memory query processing engine supporting analytical query workloads.

References

[1]

S. R Agrawal, S. Idicula, A. Raghavan, E. Vlachos, V. Govindaraju, V. Varadarajan, C. Balkesen, G. Giannikis, C. Roth, N. Agarwal, and E. Sedlar. 2017. A Many-core Architecture for In-memory Data Processing. In MICRO (MICRO '17). 245--258.

Digital Library

[2]

K. Aingaran, S. Jairath, G. K. Konstadinidis, S. Leung, P. Loewenstein, C. McAllister, S. Phillips, Z. Radovic, R. Sivaramakrishnan, D. Smentek, and T. Wicki. 2015. M7: Oracle's Next-Generation Sparc Processor. MICRO 35, 2 (2015), 36--45.

Digital Library

[3]

O. Arnold, S. Haas, G. Fettweis, B. Schlegel, T. Kissinger, and W. Lehner. 2014. An Application-specific Instruction Set for Accelerating Set-oriented Database Primitives. In SIGMOD.

Digital Library

[4]

P. Bakkum and K. Skadron. 2010. Accelerating SQL database operations on a GPU with CUDA. In GPGPU. 94--103.

Digital Library

[5]

C. Balkesen, G. Alonso, J. Teubner, and M. T. Özsu. 2013. Multi-core, Mainmemory Joins: Sort vs. Hash Revisited. Proc. VLDB Endow. 7, 1 (Sept. 2013), 85--96.

Digital Library

[6]

F. Beier and K. Stolze. 2017. Architecture of a data analytics service in hybrid cloud environments. it - Information Technology 59, 3 (2017), 151--158.

[7]

P. A. Boncz, M. Zukowski, and N. Nes. 2005. MonetDB/X100: Hyper-Pipelining Query Execution. In CIDR, Vol. 5. 225--237.

[8]

H. Boral and D. J. DeWitt. 1983. Database Machines: An Idea Whose Time Passed? A Critique of the Future of Database Machines. In Database Machines, International Workshop. 166--187.

[9]

G. Candea, N. Polyzotis, and R. Vingralek. 2009. A Scalable, Predictable Join Operator for Highly Concurrent Data Warehouses. PVLDB 2, 1 (2009), 277--288.

Digital Library

[10]

Ç. Balkesen, J. Teubner, G. Alonso, and M. T. Özsu. 2015. Main-Memory Hash Joins on Modern Processor Architectures. IEEE Trans. on Knowl. and Data Eng. 27, 7 (2015), 1754--1766.

Digital Library

[11]

E. S. Chung, J. D. Davis, and J. Lee. 2013. LINQits: Big Data on Little Clients. In ISCA (ISCA '13). 261--272.

Digital Library

[12]

David J. DeWitt. 1978. DIRECT - a Multiprocessor Organization for Supporting Relational Data Base Management Systems. In Proceedings of the 5th Annual Symposium on Computer Architecture (ISCA '78). 182-189.

Digital Library

[13]

D. J. Dewitt, S. Ghandeharizadeh, D. A. Schneider, A. Bricker, H. I. Hsiao, and R. Rasmussen. 1990. The Gamma Database Machine Project. IEEE Trans. on Knowl. and Data Eng. 2, 1 (1990), 44--62.

Digital Library

[14]

J. Do, Y. Kee, J. M. Patel, C. Park, K. Park, and D. J. DeWitt. 2013. Query Processing on Smart SSDs: Opportunities and Challenges. In SIGMOD (SIGMOD '13). ACM, 1221--1230.

Digital Library

[15]

M. Ferdman, A. Adileh, O. Kocberber, S. Volos, M. Alisafaee, D. Jevdjic, C. Kaynak, A. D.l Popescu, A. Ailamaki, and B. Falsafi. 2012. Clearing the Clouds: A Study of Emerging Scale-out Workloads on Modern Hardware. SIGPLAN Not. (2012), 37--48.

Digital Library

[16]

G. Giannikis, G. Alonso, and D. Kossmann. 2012. SharedDB: Killing One Thousand Queries with One Stone. PVLDB 5, 6 (Feb. 2012), 526--537.

Digital Library

[17]

B. Gold, A. Ailamaki, L. Huston, and B. Falsafi. 2005. Accelerating Database Operators Using a Network Processor. In DaMoN.

Digital Library

[18]

N. K. Govindaraju and D. Manocha. 2005. Efficient Relational Database Management using Graphics Processors. In DaMoN.

Digital Library

[19]

S. Haas, O. Arnold, B. Nöthen, S. Scholze, G. Ellguth, A. Dixius, S. Höppner, S. Schiefer, S. Hartmann, S. Henker, T. Hocker, J. Schreiter, H. Eisenreich, J. Schlüßler, D. Walter, T. Seifert, F. Pauls, M. Hasler, Y. Chen, H. Hensel, S. Moriam, E. Matús, C. Mayr, R. Schüffny, and G. P. Fettweis. 2016. An MPSoC for energy-efficient database query processing. In DAC. 112:1--112:6.

Digital Library

[20]

S. Harizopoulos, V. Shkapenyuk, and A. Ailamaki. 2005. QPipe: A Simultaneously Pipelined Relational Query Engine. In SIGMOD (SIGMOD '05). 383--394.

Digital Library

[21]

T. Hayes, O. Palomar, O. S. Unsal, A. Cristal, and M. Valero. 2012. Vector Extensions for Decision Support DBMS Acceleration. In MICRO '12. IEEE Computer Society, 166--176.

Digital Library

[22]

B. He, M. Lu, K. Yang, R. Fang, N. K. Govindaraju, Q. Luo, and P. V. Sander. 2009. Relational Query Coprocessing on Graphics Processors. ACM Trans. Database Syst. 34, 4 (2009), 21:1--21:39.

Digital Library

[23]

Z. István, G. Alonso, M. Blott, and K. A. Vissers. 2013. A flexible hash table design for 10GBPS key-value stores on FPGAS. In FPL. 1--8.

[24]

T. Kaldewey, G. M. Lohman, R. Müller, and P. B. Volk. 2012. GPU join processing revisited. In DaMoN. 55--62.

Digital Library

[25]

O. Kocberber, B. Grot, J. Picorel, B. Falsafi, K. Lim, and P. Ranganathan. 2013. Meet the Walkers: Accelerating Index Traversals for In-memory Databases. In MICRO (MICRO '13).

Digital Library

[26]

I. Müller, C. Ratsch, and F. Färber. 2014. Adaptive String Dictionary Compression in In-Memory Column-Store Database Systems. In EDBT. 283--294.

[27]

R. Müller, J. Teubner, and G. Alonso. 2009. Streams on Wires - A Query Compiler for FPGAs. PVLDB 2, 1 (2009), 229--240.

Digital Library

[28]

M. Najafi, M. Sadoghi, and H.-A. Jacobsen. 2013. Flexible Query Processor on FPGAs. PVLDB 6, 12 (2013), 1310--1313.

Digital Library

[29]

A. Pavlo, E. Paulson, A. Rasin, D. J. Abadi, D. J. DeWitt, S. Madden, and M. Stonebraker. 2009. A Comparison of Approaches to Large-scale Data Analysis. In SIGMOD. 165--178.

Digital Library

[30]

W. Rödiger, S. Idicula, A. Kemper, and T. Neumann. 2016. Flow-Join: Adaptive skew handling for distributed joins over high-speed networks. In ICDE. 1194- 1205.

[31]

B. Sukhwani, H. Min, M. Thoennes, P. Dube, B. Iyer, B. Brezzo, D. Dillenberger, and S. W. Asaad. 2012. Database analytics acceleration using FPGAs. In PACT. 411--420.

Digital Library

[32]

L. Wang, J. Zhan, C. Luo, Y. Zhu, Q. Yang, Y. He, W. Gao, Z. Jia, Y. Shi, S. Zhang, C. Zheng, G. Lu, K. Zhan, X. Li, and B. Qiu. 2014. BigDataBench: A big data benchmark suite from internet services. HPCA (2014), 488--499.

[33]

L. Woods, Z. István, and G. Alonso. 2014. Ibex - An Intelligent Storage Engine with Support for Advanced SQL Off-loading. PVLDB 7, 11 (2014), 963--974.

Digital Library

[34]

H. Wu, G. . Diamos, S. Cadambi, and S. Yalamanchili. 2012. Kernel Weaver: Automatically Fusing Database Primitives for Efficient GPU Computation. In MICRO. 107--118.

Digital Library

[35]

L. Wu, R. J. Barker, M. A. Kim, and K. A. Ross. 2013. Navigating Big Data with High-throughput, Energy-efficient Data Partitioning. In ISCA (ISCA '13). 249--260.

Digital Library

[36]

L. Wu, R. J Barker, M. A Kim, and K. A. Ross. 2013. Navigating Big Data with High-Throughput, Energy-Efficient Data Partitioning. In ISCA.

Digital Library

[37]

L. Wu, A. Lottarini, T. K. Paine, M. A. Kim, and K. A. Ross. 2015. The Q100 Database Processing Unit. MICRO 35 (2015).

[38]

M. Zukowski, P. Boncz, N. Nes, and S. Héman. 2005. MonetDB/X100 - A DBMS in the CPU cache. In IEEE Data Eng. Bull., Vol. 28. 17--22

Cited By

Kong HFan HWu JCheng LChen YLu WYan GLi XThapliyal HDeMara RPartin-Vaisband IKatkoori S(2023)KPU-SQL: Kernel Processing Unit for High-Performance SQL AccelerationProceedings of the Great Lakes Symposium on VLSI 202310.1145/3583781.3590268(37-43)Online publication date: 5-Jun-2023
https://dl.acm.org/doi/10.1145/3583781.3590268
Kong HLu WChen YWu JZhang YYan GLi X(2023)DOE: database offloading engine for accelerating SQL processingDistributed and Parallel Databases10.1007/s10619-023-07427-z41:3(273-297)Online publication date: 13-May-2023
https://doi.org/10.1007/s10619-023-07427-z
Guo BYu JYang DLeng HLiao B(2022)Energy-Efficient Database Systems: A Systematic SurveyACM Computing Surveys10.1145/353822555:6(1-53)Online publication date: 7-Dec-2022
https://dl.acm.org/doi/10.1145/3538225
Show More Cited By

Index Terms

RAPID: In-Memory Analytical Query Processing Engine with Extreme Performance per Watt
1. Computer systems organization
  1. Architectures
    1. Parallel architectures
      1. Multicore architectures
2. Information systems
  1. Data management systems
    1. Database management system engines
      1. Online analytical processing engines

Recommendations

A many-core architecture for in-memory data processing
MICRO-50 '17: Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture

For many years, the highest energy cost in processing has been data movement rather than computation, and energy is the limiting factor in processor design [21]. As the data needed for a single application grows to exabytes [56], there is clearly an ...
A HW/SW Co-design Methodology: An Accurate Power Efficiency Model and Design Metrics for Embedded System
SNPD '09: Proceedings of the 2009 10th ACIS International Conference on Software Engineering, Artificial Intelligences, Networking and Parallel/Distributed Computing

Low-power embedded system design has become extremely important in the most recent years. To fulfill system-level design requirements and time-to-market constraints, a power-driven methodology is essential during embedded system design. The aim of this ...
Q100: the architecture and design of a database processing unit
ASPLOS '14: Proceedings of the 19th international conference on Architectural support for programming languages and operating systems

In this paper, we propose Database Processing Units, or DPUs, a class of domain-specific database processors that can efficiently handle database applications. As a proof of concept, we present the instruction set architecture, microarchitecture, and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGMOD '18: Proceedings of the 2018 International Conference on Management of Data

May 2018

1874 pages

ISBN:9781450347037

DOI:10.1145/3183713

General Chairs:
Gautam Das
University of Texas at Arlington, USA
,
Christopher Jermaine
Rice University, USA
,
Philip Bernstein
Microsoft Research, USA

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 May 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGMOD/PODS '18

Sponsor:

SIGMOD

SIGMOD/PODS '18: International Conference on Management of Data

June 10 - 15, 2018

TX, Houston, USA

Acceptance Rates

SIGMOD '18 Paper Acceptance Rate 90 of 461 submissions, 20%;

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
1,098
Total Downloads

Downloads (Last 12 months)79
Downloads (Last 6 weeks)12

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kong HFan HWu JCheng LChen YLu WYan GLi XThapliyal HDeMara RPartin-Vaisband IKatkoori S(2023)KPU-SQL: Kernel Processing Unit for High-Performance SQL AccelerationProceedings of the Great Lakes Symposium on VLSI 202310.1145/3583781.3590268(37-43)Online publication date: 5-Jun-2023
https://dl.acm.org/doi/10.1145/3583781.3590268
Kong HLu WChen YWu JZhang YYan GLi X(2023)DOE: database offloading engine for accelerating SQL processingDistributed and Parallel Databases10.1007/s10619-023-07427-z41:3(273-297)Online publication date: 13-May-2023
https://doi.org/10.1007/s10619-023-07427-z
Guo BYu JYang DLeng HLiao B(2022)Energy-Efficient Database Systems: A Systematic SurveyACM Computing Surveys10.1145/353822555:6(1-53)Online publication date: 7-Dec-2022
https://dl.acm.org/doi/10.1145/3538225
Caminal HChronis YWu TPatel JMartínez JSalapura VZahran MChong FTang L(2022)Accelerating database analytic query workloads using an associative processorProceedings of the 49th Annual International Symposium on Computer Architecture10.1145/3470496.3527435(623-637)Online publication date: 18-Jun-2022
https://dl.acm.org/doi/10.1145/3470496.3527435
Falsafi B(2021)Post‐Moore Datacenter Server ArchitectureMulti‐Processor System‐on‐Chip 210.1002/9781119818410.ch6(123-134)Online publication date: 28-Apr-2021
https://doi.org/10.1002/9781119818410.ch6
Xu SBourgeat THuang TKim HLee SArvind A(2020)AQUOMAN: An Analytic-Query Offloading Machine2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO50266.2020.00041(386-399)Online publication date: Oct-2020
https://doi.org/10.1109/MICRO50266.2020.00041
Chugh ASharma VJain C(2020)Big Data and Query Optimization TechniquesAdvances in Computing and Intelligent Systems10.1007/978-981-15-0222-4_30(337-345)Online publication date: 3-Jan-2020
https://doi.org/10.1007/978-981-15-0222-4_30
Mustard CRuffy FGakhokidze ABeschastnikh IFedorova ADelimitrou CPorts D(2019)JumpgateProceedings of the 11th USENIX Conference on Hot Topics in Cloud Computing10.5555/3357034.3357042(6-6)Online publication date: 8-Jul-2019
https://dl.acm.org/doi/10.5555/3357034.3357042

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten