Inverted List Caching for Topical Index Shards
نویسندگان
چکیده
Selective search is a distributed retrieval architecture that intentionally creates skewed postings and access patterns. This work shows that the well-known QtfDf inverted list caching algorithm is as effective with topicallypartitioned indexes as it is with randomly-partitioned indexes. It also shows that a mixed global-local strategy reduces total I/O without harming query hit rates.
منابع مشابه
Does Selective Search Benefit from WAND Optimization?
Selective search is a distributed retrieval technique that reduces the computational cost of large-scale information retrieval. By partitioning the collection into topical shards, and using a resource selection algorithm to identify a subset of shards to search, selective search allows retrieval effectiveness to be maintained while evaluating fewer postings, often resulting in 90+% reductions i...
متن کاملStanding on the Shoulders of Peers: Caching in Peer-to-Peer Information Retrieval
State-of-the-art P2P IR systems suffer from their lack of response time guarantees, especially with scale. To address this issue, a number of techniques for caching of multi-term inverted list intersections and query results have been proposed recently. Although these enable speedy query evaluations with low network overhead, they fail to consider the potential impact of caching on result quali...
متن کاملAn Inverted File Cache for Fast Information Retrieval
The inverted file is the most popular indexing mechanism used for document search in an information retrieval system (IRS). However, the disk I/O for accessing the inverted file becomes a bottleneck in an IRS. To avoid using the disk I/O, we propose a caching mechanism for accessing the inverted file, called the inverted file cache (IF cache). In this cache, a proposed hashing scheme using a li...
متن کاملEfficient and Effective Large-scale Search
Search engine indexes for large document collections are often divided into shards that are distributed across multiple computers and searched in parallel to provide rapid interactive search. Typically, all index shards are searched for each query. For organizations with modest computing resources the high query processing cost of this exhaustive search setup can be a deterrent to working with ...
متن کاملImproved Skips for Faster Postings List Intersection
Information retrieval can be achieved through computerized processes by generating a list of relevant responses to a query. The document processor, matching function and query analyzer are the main components of an information retrieval system. Document retrieval system is fundamentally based on: Boolean, vector-space, probabilistic, and language models. In this paper, a new methodology for mat...
متن کامل