Distributing the Data Plane for Remote Storage Access

Sub-microsecond network and memory latencies require fast user-level access to local and remote storage. While user-level access to local storage has been demonstrated recently, it does currently not extend to serverless parallel systems in datacenter environments. We propose direct user-level access to remote storage in a distributed setting, unifying fast data access and high-performance remote memory access programming. We discuss a minimal hardware extension of the IOMMU to enable direct remote storage access. In order to maintain optimal performance in the system, we use epoch-based accesses to allow fine-tuning of atomicity, consistency, isolation, and durability semantics. We also address the problem of user-managed coherent caching. Finally, we briefly discuss the design of DiDAFS, a Distributed Direct Access File System that enables efficient data analytics use-cases such as buffered producer-consumer synchronization and key-value stores as well as deeper integration of storage into high performance computing applications.