Nonuniform Cache Architectures for Wire-Delay Dominated On-Chip Caches

0272-1732/03/$17.00  2003 IEEE Published by the IEEE computer Society The next generation of today’s highperformance processors incorporate large leveltwo caches on the processor die. For example, the IBM Power5 will contain a 1.92-Mbyte L2 cache, the Hewlett-Packard PA8700 will contain 2.25 Mbytes of unified on-chip cache, and the Intel Itanium2 will contain 6 Mbytes of on-chip L3 cache. Cache sizes will continue to increase as bandwidth demands on the package grow, and as smaller technologies permit more bits per square millimeter. However, increasing global wire delays across the chip will make large on-chip caches with a single, discrete hit latency undesirable in future technologies. Data residing near the processor in a large cache is much more quickly accessible than data residing far from the processor. Accessing the closest bank in a 16-Mbyte, onchip L2 cache built in a 50-nm process technology, for example, could take four cycles, whereas accessing the farthest bank might take 47 cycles. The bulk of the access time involves routing to and from the banks rather than the bank accesses themselves. Nonuniform cache access (NUCA) designs address this wire-delay problem. In this approach, a switched network allows data to migrate to different cache regions according to access frequency—that is, frequently accessed data migrates to areas closer to the processor. We propose several designs that treat the cache as a network of banks and facilitate nonuniform accesses to different physical regions. NUCA architectures offer low-latency access, increased scalability, and greater performance stability than conventional uniform access cache architectures.