Abstract
Modern applications demand high performance and cost efficient database management systems (DBMSs). Their workloads may be diverse, ranging from online transaction processing to analytics and decision support. The cloud infrastructure enables disaggregation of monolithic DBMSs into components that facilitate software-hardware co-design. This is realized using pools of hardware resources, i.e., CPUs, GPUs, memory, FPGA, NVM, etc., connected using high-speed networks. This disaggregation trend is being adopted by cloud DBMSs because hardware re-provisioning can be achieved by simply invoking software APIs. Disaggregated DBMSs separate processing from storage, enabling each to scale elastically and independently. They may disaggregate compute usage based on functionality, e.g., compute needed for writes from compute needed for queries and compute needed for compaction. They may also use disaggregated memory, e.g., for intermediate results in a shuffle or for remote caching. The DBMS monitors the characteristics of a workload and dynamically assembles its components that are most efficient and cost effective for the workload. This paper is a summary of a panel session that discussed the capability, challenges, and opportunities of these emerging DBMSs and disaggregated hardware systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
A DMS includes traditional relational database management systems, key-value stores, document stores, etc.
References
Borthakur, D.: The aggregator leaf tailer architecture (2019). https://rockset.com/blog/aggregator-leaf-tailer-an-architecture-for-live-analytics-on-event-streams/
DeWitt, D.J., Ghandeharizadeh, S., Schneider, D.A., Bricker, A., Hsiao, H.I., Rasmussen, R.: The Gamma database machine project. In: IEEE Transactions on Knowledge and Data Engineering, vol. 1(2), March 1990
Dong, S., Callaghan, M., Galanis, L., Borthakur, D., Savor, T., Strum, M.: Optimizing space amplification in RocksDB. In: 8th Biennial Conference on Innovative Data Systems Research (CIDR 2017), Chaminade, CA (2017). https://www.cidrdb.org/
Gu, J., et al.: Efficient memory disaggregation with Infiniswap. In: 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 2017), pp. 649–667. USENIX Association, Boston, MA (2017)
Facebook: Multifeed (2015). https://engineering.fb.com/2015/03/10/production-engineering/serving-facebook-multifeed-efficiency-performance-gains-through-redesign/
Ghandeharizadeh, S., Huang, H., Nguyen, H.: Nova: diffused database processing using clouds of components [vision paper]. In: Kozielski, S., Mrozek, D., Kasprowski, P., Małysiak-Mrozek, B., Kostrzewa, D. (eds.) BDAS 2019. CCIS, vol. 1018, pp. 3–14. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-19093-4_1
Ghandeharizadeh, S., Irani, S., Lam, J.: On configuring a hierarchy of storage media in the age of NVM. In: 2018 IEEE 34th International Conference on Data Engineering (ICDE), pp. 1380–1383. IEEE (2018)
Ghemawat, S., Dean, J.: LevelDB (2022). https://github.com/google/leveldb
Gupta, A.: Linkedin followfeed (2016). https://engineering.linkedin.com/blog/2016/03/followfeed-linkedin-s-feed-made-faster-and-smarter
Huang, H., Ghandeharizadeh, S.: An evaluation of RDMA-based message passing protocols. In: 2019 IEEE International Conference on Big Data, pp. 3854–3863. IEEE (2019)
Huang, H., Ghandeharizadeh, S.: Nova-LSM: a distributed, component-based LSM-tree key-value store. In: Proceedings of the 2021 International Conference on Management of Data, pp. 749–763 (2021)
Menon, J.: Next generation storage will be built with DPUs. Flash Memory Summit (2022)
Menon, J.: Next generation storage will use DPUs instead of CPUs. In: Storage Developer Conference (2022)
Noureddine, W.: The fungible DPU: a new category of microprocessor (2021). https://lp.fungible.com/hubfs/Assets/Whitepapers/The-Fungible-DPU-A-New-Category-of-Microprocessor.pdf
Executive Office of the President of the United States Machine Learning, Artificial Intelligence Subcommittee of the National Science, and Technology Council. Lessons Learned from Federal Use of Cloud Computing to Support Artificial Intelligence Research and Development (2022). https://www.whitehouse.gov/wp-content/uploads/2022/07/07-2022-Lessons-Learned-Cloud-for-AI-July2022.pdf
Rockset: Whitepaper (2022). https://rockset.com/whitepapers/rockset-concepts-designs-and-architecture/
Shan, Y., Huang, Y., Chen, Y., Zhang, Y.: LegoOS: a disseminated, distributed OS for hardware resource disaggregation. In: 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2018), pp. 69–87 (2018)
Stonebraker, M.: The case for shared nothing. Database Engineering (1986)
Zhang, Q., Bernstein, P.A., Berger, D.S., Chandramouli, B.: Redy: remote dynamic memory cache. Proc. VLDB Endow. 15(4), 766–779 (2021)
Zhang, Q., Bernstein, P.A., Berger, D.S., Chandramouli, B., Liu, V., Loo, B.T.: CompuCache: remote computable caching using spot VMs. In: Annual Conference on Innovative Data Systems Research (CIDR 2022) (2022)
Acknowledgments
We thank Liqid’s Bob Brumfield and George Wagner for input on Sect. 2.2.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ghandeharizadeh, S., Bernstein, P.A., Borthakur, D., Huang, H., Menon, J., Puri, S. (2023). Disaggregated Database Management Systems. In: Nambiar, R., Poess, M. (eds) Performance Evaluation and Benchmarking. TPCTC 2022. Lecture Notes in Computer Science, vol 13860. Springer, Cham. https://doi.org/10.1007/978-3-031-29576-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-29576-8_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-29575-1
Online ISBN: 978-3-031-29576-8
eBook Packages: Computer ScienceComputer Science (R0)