Derivation of Scalable Message-Passing Algorithms Using Parallel Combinatorial List Generator Functions

We present the transformational derivations of several efficient, scalable, message-passing parallel algorithms from clear functional specifications. The starting algorithms rely on some commonly used combinatorial list generator functions such as tails, inits, splits and cp (Cartesian product) for generating useful intermediate results. This paper provides generic parallel algorithms for efficiently implementing a small library of useful combinatorial list generator functions. It also provides a framework for relating key higher order functions such as map, reduce, and scan with communicating processes with different configurations. The parallelisation of many interesting functional algorithms can then be systematically synthesized by taking an “off the shelf” parallel implementation of the list generator and composing it with appropriate parallel implementations of instances of higher order functions. Efficiency in the final message-passing algorithms is achieved by exploiting data parallelism, for generating the intermediate results in parallel; and functional parallelism, for processing intermediate results in stages such that the output of one stage is simultaneously input to the next one. This approach is then illustrated with a number of case studies which include: testing whether all the elements of a given list are distinct, the maximum segment sum problem, the minimum distance of two sets of points, and rank sort. In each case we progress from a quadratic time initial functional specification of the problem to a linear time parallel message-passing implementation which uses a linear number of communicating sequential processes. Bird-Meertens Formalism is used to concisely carry out the transformations.