ZZ: Cheap Practical BFT using Virtualization

Despite numerous efforts to improve their performance and scalability, Byzantine fault-tolerance (BFT) techniques remain expensive, and few commercial systems use BFT today. We present ZZ, a novel approach to construct general BFT services with a replication cost of practically f + 1, halving the 2f + 1 or higher cost incurred by state-of-the-art approaches. The key insight in ZZ is to use f + 1 execution replicas in the normal case and to activate additional replicas only upon failures. ZZ uses virtual machines for fast replica activation and several novel mechanisms for rapid recovery of these replicas such as using filesystem snapshots to reduce checkpointing overhead, replaying state updates instead of full requests, and an amortized state transfer mechanism that fetches state on-demand. We have implemented ZZ using the BASE library, Xen virtual machines and the ZFS file system. Our experimental evaluation shows that the recovery time of ZZ replicas is independent of the application disk state, taking less than 4s for 400MB of disk state, at the expense of a small increase in request latency in the fault-mode.