ECS 201A Survey Recent Advances in Caching Techniques

Replacement and Insertion policies play a key part in determining the effectiveness of caching in both uni-processor and multi-processor systems. General strategies have been designed to take advantage of the principle of locality obeyed by a large class of computer programs-Temporal Locality and Spatial Locality. On this note, the LRU replacement policy and its variations have been the de-facto standard in processor systems, but it has several shortcomings. Performance of LRU is severely affected when there is little locality in the behavior of the application program. To overcome this, some novel replacement strategies have been proposed that can fare well in both LRU-friendly and LRU-averse scenarios. We take a look at two of them-Adaptive Insertion Policies [15] and Re-Reference Interval Prediction (RRIP) [5]. Electrical interconnects are viewed as a major limiting factor with respect to designing high performance caches, with regard to latency, bandwidth, and power. Several hybrid architectures have been proposed which sought to provide lower average latency, lower cost and better overall performance. The increasing sizes of on-chip caches and the growing domination of wire delay necessitate significant changes to cache hierarchy design methodologies. To solved the scal-ability issues in multi-banked caches which form a Uniform Cache Architecture (UCA), non-uniform cache architectures (NUCA) have been proposed, where the banks are connected with an interconnect fabric and access time for a block is a function of the delays experienced in traversing the network path from the cache controller to the bank that contains the block. We take a look at Interconnect Design Delays for large NUCA caches [12], Reactive NUCA [4], PageNUCA [3] and ESP-NUCA [11]. Modern multi-core processors employ large last-level caches (LLC) shared between all of the cores. Researchers have proposed a variety of techniques to manage the LLC to provide better performance and fairness. Most of these schemes follow a pattern of observation, policy selection, and enforcement. Caches for multi-core processors can be managed both by hardware and software. Here we focus on two novel proposals of both mechanisms. Improvements in computational capability attained by Moore's law and microprocessor architectural innovations are over time balanced by corresponding improvements in I/O capability. 10GbE connectivity is expected to be a standard feature of server platforms in the near future. Among the numerous methods and features proposed to improve network performance of such platforms is Direct Cache Access (DCA) to route incoming I/O to CPU caches directly. We take …