Improvement of energy-efficiency in off-chip caches by selective prefetching

In this paper we revisit the line size/performance trade-offs in off-chip second-level caches in light of energy efficiency and make two distinct contributions: Our first observation is that while large blocks, i.e., 128-256 bytes, typically improve performance, they cause a devastating energy dissipation because the limited spatial locality results in a low block utilization. We find that blocks as small as 32 bytes can cut the energy-delay (the execution time/energy used product) by more than a factor of two with only a moderate performance loss that is less than 25%. As one technique to exploit spatial locality, we apply selective software-controlled prefetching to compensate for the moderate performance losses of small blocks. Aided by a performance-tuning tool (in our case study we use SimICS), we identify the memory instructions that contribute to most of the misses. In the seven applications we have studied, typically less than ten memory instructions cause half the number of misses. By inserting prefetch instructions to cover their misses, we show that it is possible to arrive at the performance level of large blocks using a small block. Moreover, the combined effect of prefetching using small blocks yield the same low energy dissipation as small blocks with no prefetching.