The Utility of Exploiting Idle Memory for Data-Intensive Computations

In this paper, we examine the utility of exploiting idle memory in workstation pools. We attempt to answer the following questions. First, given a workstation pool, what fraction of the memory can be expected to be idle? This provides an estimate of the opportunity for hosting guest data. Second, what fraction of a individual host' s memory can be expected to be idle? This helps determine the recruitment policy – what is the maximum amount of memory that should be recruited on a single host? Third, what is the distribution of memory idle-times? That is, what is the probability that a chunk of memory that is currently idle will be idle for longer than time t? This information indicates how long guest data can be expected to survive; applications that access their data-sets frequently within the expected life-time of guest data are more likely to benefit from exploiting idle memory. Fourth, how much benefit can a user expect? We use two metrics for the benefit of exploiting idle memory: (1) if I have a pool with w workstations, how much memory should I expect to get for free by harvesting idle memory; (2) how much improvement can be achieved in end-to-end execution time? Finally, how long and how frequently might a user have to wait to reclaim her machine if she volunteers to host guest pages on her machine? This helps answer the question of social acceptability. To answer the questions relating to the availability of idle memory, we have analyzed two-week long traces from five workstation pools with different sizes, locations, and patterns of use. To evaluate the expected benefits and costs, we have simulated three data-intensive applications (0.5 GB-5 GB) on these workstation pools.