Memory systems in the many-core era: challenges, opportunities, and solution directions

The memory subsystem is a fundamental performance and energy bottleneck in almost all computing systems. Recent trends towards increasingly more cores on die, consolidation of diverse workloads on a single chip, and difficulty of DRAM scaling impose new requirements and exacerbate old demands on the memory system. In particular, the need for memory bandwidth and capacity is increasing [14], applications' interference in memory system increasingly limits system performance and makes the system hard to control [12], memory energy and power are key design concerns [8], and DRAM technology consumes significant amount of energy and does not scale down easily to smaller technology nodes [7]. Fortunately, some promising solution directions exist.

In this talk, we will examine recent technology, application, and architecture trends motivating a fundamental rethinking of the memory hierarchy. Based on this motivation, we will describe requirements from an ideal memory system suitable for the many-core era. The talk will examine questions one would need to answer in approximating the ideal memory system and possible avenues that seem promising for the research community to explore. In particular, we will focus on the problem of uncontrolled inter-application interference in the memory system and draw upon our experiences in solving it by designing quality-of-service (QoS) aware memory controllers [5, 6, 9, 10, 11, 12], interconnects [1 2 13], and entire memory systems. We will make a case forapplication- and QoS-aware design of memory systems and [3, 4]integrated/cooperative design of cores, interconnects, and memory components to optimize the overall system.