Investigation of Load Balancing Scalability in Space Plasma Simulations

This whitepaper describes the load-balancing performance issues that are observed and tackled during the petascaling of a space plasma simulation code developed at the Finnish Meteorological Institute (FMI). It models the communication pattern as a hypergraph, and partitions the computational grid using the parallel hypergraph partitioning scheme (PHG) of the Zoltan partitioning framework. The result of partitioning determines the distribution of grid cells to processors. It is observed that the partitioning phase takes a substantial percentage of the overall computation time. Alternative (graph-partitioning-based) schemes that perform almost as well as the hypergraph partitioning scheme and that require less preprocessing overhead and better balance are proposed and investigated. A comparison in terms of effect on running time, preprocessing overhead and loadbalancing quality of Zoltan’s PHG, ParMeTiS, and PT-SCOTCH are presented. Test results on Jüelich BlueGene/P cluster are presented.