Designing a Storage Infrastructure for Scalable Cloud Services

Abstract. In an IaaS (Infrastructure-as-a-Service) cloud services, storage needs of VM (Virtual Machine) instances are met through virtual disks (i.e. virtual block devices). However, it is nontrivial to provide virtual disks to VMs in an efficient and scalable way for a couple of reasons. First, a VM host may be required to provide virtual disks for a large number of VMs. It is difficult to ascertain the largest possible storage demands and physically provision them all in the host machine. On the other hand, if the storage spaces for virtual disks are provided through remote storage servers, aggregate network traffic due to storage accesses from VMs can easily deplete the network bandwidth and cause congestion. We propose a system, vStore, which overcomes these issues by utilizing the host’s limited local disk space as a block-level cache for the remote storage in order to absorb network traffics from storage accesses. This allows the VMM (Virtual Machine Monitor) to serve VMs’ disk I/O requests from the host’s local disks most of the time, while providing the illusion of much larger storage space for creating new virtual disks. Caching is a well-studied topic in many different contexts, but caching virtual disks at block-level poses special challenges in achieving high performance while maintaining virtual disk semantics. First, after a disk write operation finishes from the VM’s perspective, the data should survive even if the host immediately encounters a power failure. Second, as disk I/O performance is dominated by disk seek times, it is important to keep a virtual disk as sequential as possible in the limited cache space. Third, the destaging operation that sends dirty pages back to the remote storage server should be self-adaptive and minimize the impact on the foreground traffic. We propose techniques to address these challenges and implemented them in Xen. Our evaluation shows that vStore provides the illusion of unlimited storage space, significantly reduces network traffic, and incurs a low disk I/O performance overhead.