PyMW - a Python Module for Parallel Master Worker Computing
نویسندگان
چکیده
We describe a general purpose master-worker parallel computation Python module called PyMW. PyMW provides a unified interface to multiple computation environments including multicore processors, networked clusters and the Berkeley Open Infrastructure for Network Computing (BOINC) software platform. PyMW is intended to support rapid development, testing and deployment of large scale master-worker style computations. It is also designed to allow easy extension to other computing environments with little change in the master-worker program. We demonstrate the effectiveness and scalability of PyMW by performing several master-worker style parallel computations on a multicore machine, a networked cluster and a BOINC project.
منابع مشابه
Work Queue + Python: A Framework For Scalable Scientific Ensemble Applications
Even with the increase in the number and variety of computer resources available to research scientists today, it is still challenging to construct scalable distributed applications. To address this issue, we developed Work Queue, a flexible master/worker framework for building large scale scientific ensemble applications that span many machines including clusters, grids, and clouds. In this pa...
متن کاملMYMPI - MPI programming in Python
We introduce an MPI Python module, MYMPI, for parallel programming in Python using the Message Passing Interface (MPI). This is a true Python module which runs with a standard Python interpreter. In this paper we discuss the motivation for creating the MYMPI module, along with differences between MYMPI and pyMPI, another MPI Python interpreter. Additionally, we discuss three projects that have ...
متن کاملUsing Mobile Agents for Parallel Processing
Mobile agents are a promising model for distributed computing and has been exploited in several areas of applications. One of those areas that may benefit from the use of mobile agent technology is parallel processing. This paper describes a Java-based platform that provides some support for parallel computing. We have implemented a special module to support the well-known Master/Worker model a...
متن کاملA GRASS GIS parallel module for radio-propagation predictions
Geographical information systems are ideal candidates for the application of parallel programming techniques, mainly because they usually handle large data sets. To help us deal with complex calculations over such data sets, we investigated the performance constraints of a classic master-worker parallel paradigm over a message-passing communication model. To this end, we present a new approach ...
متن کاملDistributed Computing with Hierarchical Master-worker Paradigm for Parallel Branch and Bound Algorithm
This paper discusses the impact of the hierarchical master-worker paradigm on performance of an application program, which solves an optimization problem by a parallel branch and bound algorithm on a distributed computing system. The application program, which this paper addresses, solves the BMI Eigenvalue Problem, which is an optimization problem to minimize the greatest eigenvalue of a bilin...
متن کامل