Distributed Multisearch and Resource Selection for the TREC Million Query Track

A distributed information retrieval system with resource‐selection and result‐set merging capability was used to search subsets of the GOV2 document corpus for the 2008 TREC Million Query Track. The GOV2 collection was partitioned into host‐name subcollections and distributed to multiple remote machines. The Multisearch demonstration application restricted each search to a fraction of the available sub‐collections that was pre‐determined by a resource‐selection algorithm. Experiment results from topic‐by‐topic resource selection and aggregate topic resource selection are compared. The sensitivity of Multisearch retrieval performance to variations in the resource selection algorithm is discussed. The information processing research group at ARSC works on problems affecting the performance of distributed information retrieval applications such as metasearch [1], federated search [2], and collection sampling [3]. An ongoing goal of this research is to guide the selection of standards and reference implementations for Grid Information Retrieval (GIR) applications [4]. Prototype GIR applications developed at ARSC help to evaluate theoretical research and gain experience with the capabilities and limitations of existing APIs, middleware, and security requirements. The TREC experiments provide an additional context to test and develop distributed IR technology. Prior TREC Terabyte (TB) and Million Query (MQ) Track experiments performed at ARSC have explored the IR performance and search efficiency of result‐set merging and ranking across small numbers of heterogeneous systems and large numbers of homogeneous systems. In the 2005 TREC [5] TB Track [6], the ARSC IR group used a variant of the logistic regression merging strategy [7], modified for efficiency, to merge results from a metasearch‐ style application that searched and merged results from two indexed copies of the GOV2 corpus [8]. One index was constructed with the Lucene Toolkit and the other index was constructed with the Amberfish application [9]. For the 2006 TREC [10] TB Track [11], the GOV2 corpus was partitioned into approximately 17,000 collections by grouping documents with identical URL host names [12]. Each query was searched against every collection and the ranked results from each collection were merged using the logistic regression algorithm used in the 2005 TB track. The large number of document collections used in the 2006 experiment, coupled with the distribution of collection size (measured in number of documents contained) that spanned five orders of magnitude, introduced significant wall‐clock, bandwidth, and IR performance problems. Follow‐up work found that the bandwidth performance could be improved somewhat without sacrificing IR performance by