skip to main content
column

The case for RAMClouds: scalable high-performance storage entirely in DRAM

Published: 27 January 2010 Publication History

Abstract

Disk-oriented approaches to online storage are becoming increasingly problematic: they do not scale gracefully to meet the needs of large-scale Web applications, and improvements in disk capacity have far outstripped improvements in access latency and bandwidth. This paper argues for a new approach to datacenter storage called RAMCloud, where information is kept entirely in DRAM and large-scale systems are created by aggregating the main memories of thousands of commodity servers. We believe that RAMClouds can provide durable and available storage with 100-1000x the throughput of disk-based systems and 100-1000x lower access latency. The combination of low latency and large scale will enable a new breed of dataintensive applications.

References

[1]
Andersen, D., Franklin, J., Kaminsky, M., et al., "FAWN: A Fast Array of Wimpy Nodes", Proc. 22nd Symposium on Operating Systems Principles, 2009, to appear.
[2]
Arista Networks 7100 Series Switches, http://www.aristanetworks.com/en/7100Series.
[3]
Armbrust, M., Fox, A., Griffith, R., et al., Above the Clouds: A Berkeley View of Cloud Computing, Technical Report UCB/EECS-2009-28, Electrical Engineering and Computer Sciences, U.C. Berkeley, February 10, 2009, http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf.
[4]
Chang, F., Dean, J, Ghemawat, S., et al., "Bigtable: A Distributed Storage System for Structured Data", ACM Transactions on Computer Systems, Vol. 26, No. 2, 2008, pp. 4:1--4:26.
[5]
Chun, B., Mainwaring, A., and Culler, D., "Virtual Network Transport Protocols for Myrinet," IEEE Micro, Vol. 18, No. 1 (January 1998), pp. 53--63.
[6]
Cooper, B., Ramakrishnan, R., Srivastava, U., et. al., "PNUTS: Yahoo!'s Hosted Data Serving Platform," VLDB '08, Proc. VLDB Endowment, Vol. 1, No. 2, (2008), pp. 1277--1288.
[7]
Dean, J., and Ghemawat, S., "MapReduce: Simplified Data Processing on Large Clusters," Proc. 6th USENIX Symposium on Operating Systems Design and Implementation, 2004, pp. 137--150.
[8]
DeCandia, G., Hastorun, D., Jampani, M., et al., "Dynamo: Amazon's Highly Available Key-value Store", Proc. 21st ACM Symposium on Operating Systems Principles, October 2007, pp. 205--220.
[9]
DeWitt, D., Katz, R., Olken, F., et al., "Implementation Techniques for Made Memory Database Systems," Proc. SIGMOD 1984, pp. 1--8.
[10]
Dittia, Z., Integrated Hardware/Software Design of a High-Performance Network Interface, Ph.D. dissertation, Washington University in St. Louis, 2001.
[11]
Garcia-Molina, H., and Salem, K., "Main Memory Database Systems: An Overview," IEEE Transactions on Knowledge and Data Engineering, Vol. 4, No. 6, December 1992, pp. 509--516.
[12]
Gray, J., and Putzolu, G.F., "The Five-minute Rule for Trading Memory for Disc Accesses, and the 10 Byte Rule for Trading Memory for CPU Time," Proc. SIGMOD 1987, June 1987, pp. 395--398.
[13]
Johnson, R., and Rothschild, J., personal communications, March 24, 2009 and August 20, 2009.
[14]
Kallman, R., Kimura, H., Natkins, J., et al., "Hstore: a High-Performance, Distributed Main Memory Transaction Processing System," VLDB'08, Proc. VLDB Endowment, Vol. 1, No. 2, (2008), pp. 1496--1499.
[15]
Lowell, D., and Chen, P., "Free Transactions With Rio Vista," 16th ACM Symposium on Operating Systems Principles, October, 1997, pp. 92--101.
[16]
memcached: a distributed memory object caching system, http://www.danga.com/memcached/.
[17]
Patterson, D., Gibson, G., and Katz, R., "A Case for Redundant Arrays of Inexpensive Disks," Proc. SIGMOD 1988, June 1988, pp. 109--116.
[18]
Ramakrishnan, R., and Gehrke, J., Database Management Systems, Third Edition, McGraw-Hill, 2003.
[19]
Reuter, A., and Haerder, T., "Principles of Transaction-Oriented Database Recovery," ACM Computing Surveys, Vol. 15, No. 4, December 1983, pp. 287--317.
[20]
Robbins, S., RAM is the new disk..., http://www.infoq.com/news/2008/06/ram-is-disk.
[21]
Rosenblum, M. and Ousterhout, J., "The Design and Implementation of a Log-Structured File System," ACM Transactions on Computer Systems, Vol. 10, No. 1, February 1992, pp. 26--52.
[22]
Schroeder, B., Pinheiro, E., and Weber, W-D., "DRAM Errors in the Wild: A Large-Scale Field Study," SIGMETRICS/Performance'09, pp. 193--204.
[23]
Stonebraker, M., Madden, S., Abadi, D., et al., "The End of an Architectural Era (It's Time for a Complete Rewrite)", Proc. VLDB '07, pp. 1150--1160.

Cited By

View all
  • (2024)SLIPP: A Space-Efficient Learned Index for String KeysProceedings of the 2024 6th International Conference on Big-data Service and Intelligent Computation10.1145/3686540.3686550(69-77)Online publication date: 29-May-2024
  • (2024)CoRD: Combining Raid and Delta for Fast Partial Updates in Erasure-Coded Storage ClustersProceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis10.1109/SC41406.2024.00113(1-14)Online publication date: 17-Nov-2024
  • (2024) Reviving Storage Systems Education in the 21 st Century — An experience report 2024 IEEE 24th International Symposium on Cluster, Cloud and Internet Computing (CCGrid)10.1109/CCGrid59990.2024.00074(616-625)Online publication date: 6-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM SIGOPS Operating Systems Review
ACM SIGOPS Operating Systems Review  Volume 43, Issue 4
January 2010
105 pages
ISSN:0163-5980
DOI:10.1145/1713254
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 January 2010
Published in SIGOPS Volume 43, Issue 4

Check for updates

Qualifiers

  • Column

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)133
  • Downloads (Last 6 weeks)26
Reflects downloads up to 06 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)SLIPP: A Space-Efficient Learned Index for String KeysProceedings of the 2024 6th International Conference on Big-data Service and Intelligent Computation10.1145/3686540.3686550(69-77)Online publication date: 29-May-2024
  • (2024)CoRD: Combining Raid and Delta for Fast Partial Updates in Erasure-Coded Storage ClustersProceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis10.1109/SC41406.2024.00113(1-14)Online publication date: 17-Nov-2024
  • (2024) Reviving Storage Systems Education in the 21 st Century — An experience report 2024 IEEE 24th International Symposium on Cluster, Cloud and Internet Computing (CCGrid)10.1109/CCGrid59990.2024.00074(616-625)Online publication date: 6-May-2024
  • (2024)Rethinking the Producer-Consumer Relationship in Modern DRAM-Based SystemsIEEE Access10.1109/ACCESS.2024.351437712(196207-196239)Online publication date: 2024
  • (2024)A quantitative evaluation of persistent memory hash indexesThe VLDB Journal — The International Journal on Very Large Data Bases10.1007/s00778-023-00812-133:2(375-397)Online publication date: 1-Mar-2024
  • (2023)PatronusProceedings of the 21st USENIX Conference on File and Storage Technologies10.5555/3585938.3585958(315-330)Online publication date: 21-Feb-2023
  • (2023)GL-CacheProceedings of the 21st USENIX Conference on File and Storage Technologies10.5555/3585938.3585946(115-133)Online publication date: 21-Feb-2023
  • (2023)DDUC: an erasure-coded system with decoupled data updating and codingDDUC: 数据更新与编码解耦的纠删码系统Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.220046624:5(716-730)Online publication date: 2-Jun-2023
  • (2023)Bringing Compiling Databases to RISC ArchitecturesProceedings of the VLDB Endowment10.14778/3583140.358314216:6(1222-1234)Online publication date: 1-Feb-2023
  • (2023)MEMTIS: Efficient Memory Tiering with Dynamic Page Classification and Page Size DeterminationProceedings of the 29th Symposium on Operating Systems Principles10.1145/3600006.3613167(17-34)Online publication date: 23-Oct-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media