Using a Way Cache to Improve Performance of Set-Associative Caches

Abstract

Modern high–performance out–of–order processors use L1 caches with increasing degree of associativity to improve performance. Higher associativity is not always feasible for two reasons: it increases cache hit latency and energy consumption. One of the main reasons for the increased latency is a multiplexor delay to select one of the lines in a set. The multiplexor is controlled by a hit signal, which means that tag comparison needs to be completed before the multiplexor can be enabled. This paper proposes a new mechanism called Way Cache for setting the multiplexor ahead of time in order to reduce the hit latency. The same mechanism allows access to only one of the tag stores and only one corresponding data store per cache access, which reduces the energy consumption. Unlike way prediction, the Way Cache always contains correct way information - but has misses. The performance of Way Cache is evaluated and compared with Way Prediction for data and instruction caches. The performance of the Way Cache is also evaluated in the presence of a Cached Load/Store Queue, an integrated L0 cache-Load/Store Queue which significantly reduces the number of accesses to the L1 cache.

This work was supported in part by the National Science Foundation under grant NSF CCR-0311738.