The JXP Method for Robust PageRank Approximation in a Peer-to-Peer Web Search Network

Link analysis on Web graphs and social networks form the foundation for authority assessment, search result ranking, and other forms of Web and graph mining. The PageRank (PR) method is the most widely known member of this family. All link analysis methods perform Eigenvector computations on a potentially huge matrix that is derived from the underlying graph, and the large size of the data makes this computation very expensive. Various techniques have been proposed for speeding up these analyses by partitioning the graph into disjoint pieces and distributing the partitions among multiple computers. However, all these methods require a priori knowledge of the entire graph and careful planning of the partitioning. This paper presents the JXP algorithm for computing PR-style authority scores of Web pages that are arbitrarily distributed over many sites of a peer-to-peer (P2P) network. Peers are assumed to compile their own data collections, for example, by performing focused Web crawls according to their interest profiles. This way, the Web graph fragments that reside at different peers may overlap and, a priori, peers do not know the relationships between different fragments. Partially supported by the EU within the 6th Framework Programme under contract 001907 “Dynamically Evolving, Large Scale Information Systems” (DELIS). Josiane Xavier Parreira Max-Planck Institute for Informatics E-mail: [email protected] Carlos Castillo Yahoo! Research E-mail: [email protected] Debora Donato Yahoo! Research E-mail: [email protected] Sebastian Michel Max-Planck Institute for Informatics E-mail: [email protected] Gerhard Weikum Max-Planck Institute for Informatics E-mail: [email protected] The JXP algorithm runs at every peer, and it works by combining locally computed authority scores with information obtained from other peers by means of random meetings among the peers in the network. The computation on the combined input of two peers is based on a Markov-chain state-lumping technique, and can be viewed as an iterative approximation of global authority scores. JXP scales with the number of peers in the network. The computations at each peer are carried out on small graph fragments only, and the storage and memory demands per peer are in the order of the size of the peer’s locally hosted data. It is proven that the JXP scores converge to the true PR scores that one would obtain by a centralized PR computation on the global graph. The paper also discusses the issue of misbehaving peers that attempt to distort the global authority values by providing manipulated data in the peer meetings. An extended version of JXP, coined TrustJXP, provides a variety of countermeasures, based on statistical techniques, for detecting suspicious behavior and combining JXP rankings with reputationbased scores.