Investigation of Parallel Computations: Machines, Tools and Programs

Parallel and distributed computing are proving to be the next logical step for high performance computing. As single processor systems approach the limits of current VLSI technology, multiple processor systems will present a feasible architecture for scalable high performance computing and resource utilisation. Furthermore as workstations are becoming more diiuse and more reliant on network technology, distributed computing presents an attractive means of optimal utilisation of networked resources. In this report both multi-processor and distributed computing systems are studied in the parallelisation of the VA, Chirp, LBG and Move-Means algorithms common in digital signal processing research. Multi-processors are shown to provide a more friendly development environment due to the shared memory programming model and better eeciency due to low communication latencies. A workstation cluster is shown to be harder to use for application programmers due to the message-passing model and parallelisation eeciency is lower due to the high communication latencies. The message-passing paradigm both facilitates and enforces optimal parallelisation considerations (like data locality) at the cost of greater eeort. The Chirp and LBG algorithms parallelise well on both architectures, but the VA and Move-Means are poor candidates for parallelisation. Issues like development environment, program analysis and debugging, task allocation, migration and load balancing and resource utilisation are shown to be important factors in the use of parallel and distributed computing system.