And All of a Sudden: Main Memory Is Less Expensive Than Disk

Abstract

Even today, the wisdom for storage still is that storing data in main memory is more expensive than storing on disks. While this is true for the price per byte, the picture looks different for price per bandwidth. However, for data driven applications with high throughput demands, I/O bandwidth can easily become the major bottleneck. Comparing costs for different storage types for a given bandwidth requirement shows that the old wisdom of inexpensive disks and expensive main memory is no longer valid in every case. The higher the bandwidth requirements become, the more cost efficient main memory is. And all of sudden: main memory is less expensive than disk.

In this paper, we show that database workloads for the next generation of enterprise systems have vastly increased bandwidth requirements. These new requirement favor in-memory systems as they are less expensive when operational costs are taken into account. We will discuss mixed enterprise workloads in comparison to traditional transactional workloads and show with a cost evaluation that main memory systems can turn out to incur lower total costs of ownership than their disk-based counterparts.

Appendix

1.1 7.1 Execution of CH-benCHmark Queries

The following adaptions have been done to run the CH-benCHmark queries:

• when needed, the extract function (e.g., EXTRACT(YEAR FROM o_entry_d)) has been replaced by the year function (e.g., YEAR(o_entry_d))

• for MySQL and PostgreSQL, query 15 has been modified to use a view instead of using SQL’s having clause (code provided by the OLTP-Bench framework)

• when needed, aliases have been resolved in case they are not supported in aggregations

We set the maximal query execution time to 12 h for each query, which excludes queries from our results even though they are executable. Due to their long execution time we assume that the execution of these queries does not terminate.

1.2 7.2 TCO Calculations

The following section lists the components for an assumed bandwidth requirement of 40 GB/s. The prices have been obtained from the official websites of hardware vendors and do not include any discounts. Energy costs are calculated using the technical specifications of the hardware. Cooling costs are calculated using an assumed Power Usage Effectiveness (PUE) of 1.8 according to the Uptime Institute 2012 Data Center Survey³. The cost of energy is $0,276 per kWh. Both energy and cooling costs are calculated for a timespan of three years.

For the hard disk and solid state disk based systems each node is a four processor server (4 $\times$ Intel Xeon E7-4850v2 12C/24T 2.3 GHz 24 MB) with an estimated price of $30,000. For both configurations the size of main memory is set to ${\sim }10\,\%$ of the database volume (i.e., 50 GB for the 500 GB data set).

All following exemplary calculations do not include costs for high availability.

HDD-Based System. The HDD-based system adapts to higher bandwidth requirements by adding direct attached storage units. In this calculation, each node has eight SAS slots. Each DAS unit is connected to two SAS slots and is assumed to provide the maximal theoretical throughput of 6 GB/s and consists of 96 disks (10 K enterprise grade) to provide the bandwidth. It is possible to reach 6 GB/s with fewer 15 K disks, but a configuration with 10 K is more price efficient.

Since two SAS slots are used to connect each DAS unit, each server node can connect to a maximum of four DAS units resulting in a peak bandwidth of 24 GB/s. Consequently, any bandwidth higher than 24 GB/s requires an additional server node.

The hardware setup for the 40 GB/s configuration and its TCO calculation is listed in Sect. 7.2 (Table 2).

Table 2. TCO calculation for the HDD-based system.

SSD-Based System. The SSD-based system uses PCI-e connected solid state disks. Recent Intel Xeon CPUs have up to 32 PCI-e lanes per socket that are directly connected. Consequently, we assume a theoretical setup of up to eight PCIe-connected SSDs per server node.

For our calculations, we use an PCIe SSD that provide a peak read bandwidth of 3 GB/s and has a size of 1 TB. As of now, there are faster SSDs available (up to 6 GB/s), but these are more expensive by a factor of over 3x. We also calculated prices for another PCIe SSD vendor whose drives are almost a factor 2x less expensive in their smallest size of 350 GB. We did not include these calculations here, as these drives are currently not available.. However, even using these drives the 40 GB/s configuration is still more expensive than its main memory-based counterpart (Table 3).

Table 3. TCO calculation for the SSD-based system.

Main Memory-Based System. The main memory-based server is equipped with Intel’s latest XEON E7 CPU. A server with four CPUs (Intel Xeon E7-4890v2 15C/30T 2.8 GHz 37 MB) costs ${\sim }\$63,000$. The costs include a 600 GB enterprise-grade HDD for persistence (Table 4).

Table 4. TCO calculation for the main memory-based system.