Distributed Aggregation Schemes for Scalable Peer-to-Peer and Grid Computing

Peer-to-Peer (P2P) systems and P2P Grids are evolving as two viable distributed computing paradigms for wide-area resource sharing on the Internet. To achieve scalable distribution of processing workload among the nodes, large-scale distributed systems such as P2P Grids need to acquire some global information in a fully decentralized fashion. This paper presents efficient schemes for building distributed aggregation trees (DAT) on a structured P2P network like Chord. By leveraging the topology and routing mechanisms of Chord, the DAT trees are implicitly constructed from native Chord routing paths without membership maintenance. To balance the DAT trees, we propose a balanced routing algorithm on Chord that dynamically selects the parent of a node from its finger nodes by its distance to the root. This paper proves that this balanced routing algorithm enables the construction of almost completely balanced DATs, when nodes are evenly distributed in the Chord identifier space. We have evaluated the performance and scalability of a DAT prototype implementation with up to 8192 nodes. Our experimental results show that the balanced DAT scheme scales well to a large number of nodes and corresponding aggregation trees. Without maintaining explicit parent-child membership, it has very low overhead during node arrival and departure. We demonstrate that the DAT scheme performs well in Grid resource monitoring, P2P reputation management, distributed RDF repository, and worm signature generation systems.