System-level Optimizations for Memory Access in the Execution Migration Machine (EM2)

In this paper, we describe system-level optimizations for the Execution Migration Machine (EM2), a novel shared-memory architecture to address the memory wall and scalability issues for large-scale multicores. In EM2, data is never replicated and threads always migrate to the core where data is statically stored. This enables EM2 not only to provide cache coherence without any complex protocols or expensive directories, but also to better utilize on-chip cache and thus experience much lower cache miss rate. However, it may incur significant execution migrations for shared data, which increases memory latency and network traffic, and thus, keeping migration rates low is a key under EM2. We present systematic application optimization techniques to address this problem for EM2 suitable for a compiler/OS implementation. Applying these optimizations manually to parallel benchmarks from the SPLASH-2 suite, we dramatically reduce the average migration rate for EM2 by 53%, which directly improves parallel completion time by 34% on average. This allows EM2 to perform competitively compared to a traditional cache-coherent architecture, on a conventional electrical network.