Models and Resource Metrics for Parallel and Distributed Computation

This paper presents a framework of using resource metrics to characterize the various models of parallel computation. Our framework reeects the approach of recent models to abstract architectural details into several generic parameters, which we call resource metrics. We examine the diierent resource metrics chosen by diierent parallel models, categorizing the models into four classes: the basic synchronous models, and extensions of the basic models which more accurately reeect practical machines by incorporating notions of asynchrony, communication cost and memory hierarchy. We then present a new parallel computation model, the LogP-HMM model, as an illustration of design principles based on the framework of resource metrics. The LogP-HMM model extends an existing parameterized network model (LogP) with a sequential hierarchical memory model (HMM) characterizing each processor. The result captures both network communication costs and the eeects of multileveled memory such as local cache and I/O. More generally, the LogP-HMM is representative of a class of models formed by combining a network model with any of several existing hierarchical memory models. Along these lines we introduce a variant of the LogP-HMM model, the LogP-UMH, which combines the LogP with the Universal Memory Hierarchy (UMH) model. We examine the potential utility of both our models in the design of several near optimal FFT and sorting algorithms. We also examine the potential of the LogP-UMH to more accurately reeect parallel machines by matching the model to the CM-5 and IBM SP2.