Optimizing Data Locality between the Swift Parallel Programming System and the FusionFS Distributed File System

Many of the high-performance computing (HPC) systems use a centralized storage system that is separate from the compute system. This approach is not going to be scalable as we seek to achieve exa-scale performance[6]. Distributed file systems can provide the scalability needed for exa-scale computing. FusionFS is a file system designed for HPC systems that achieves scalability in part by removing bottlenecks found in metadata management. Swift/T is a high level, implicitly parallel scripting language for HPC systems. Swift/T provides automated parallelism and load balancing on a massive scale. Additional optimizations can be achieved by utilizing the features FusionFS and Swift/T to take advantage of locality. In this paper, we will look at using Swift/T’s language features to optimize locality in FusionFS.