Totally Ordered Multicast in Large-Scale Systems

Totally ordered multicast protocols have proved to be extremely useful in supporting fault-tolerant distributed applications. This paper compares the performance of the two main classes of protocols providing total order in large-scale systems: token-site and symmetric protocols. The paper shows that both classes of protocols can exhibit a latency close to 2D, where D is the message transit delay between two processes. In face of these observations, the paper makes the following contributions: it presents a rate-synchronization scheme for symmetric protocols that exhibits a latency close to D+ t, where t is the largest inter-message transmission time; it proposes a new hybrid protocol and shows that the hybrid scheme for heterogeneous topologies performs better than any of the previous classes of protocols in isolation; finally, the paper presents an algorithm that allows a process to dynamically adapt to changes in throughput and in network delays. The combination of these three techniques results in a dynamic hybrid scheme that, when applied to systems where the topology/traffic patterns are not known a priori, offers a much lower latency than non-hybrid approaches.