Executing Communication-Intensive Irregular Programs Efficiently

We consider the problem of eÆciently executing completely irregular, communication-intensive parallel programs. Completely irregular programs are those whose number of parallel threads as well as the amount of computation performed in each thread vary during execution. Our programs run on MIMD computers with some form of space-slicing (partitioning) and time-slicing (scheduling) support. A hardware barrier synchronization mechanism is required to eÆciently implement the frequent communications of our programs, and this constrains the computer to a xed size partitioning policy. We compare the possible scheduling policies for irregular programs on xed size partitions: local scheduling and multi-gang scheduling, and prove that local scheduling does better. Then we introduce competitive analysis and formally analyze the online rebalancing algorithms required for eÆcient local scheduling under two scenarios: with full information and with partial information.