Meta Searching the Web using Exemplar Texts: Initial Results

An Exemplar Text is the ideal model result for Web searches: It characterises what we would like our queries to find. Yet, this rarely happens, since the World Wide Web (WWW) suffers from several deficiencies as an Information Retrieval system. Firstly, the Web does not contain a document collection since the material it contains has not been gathered for one subject or purpose. It is also incompletely indexed because of its size. Consequently retrieving text from a search engine is time consuming and laborious. Hesperus is a system designed to address this problem. Using an electronic encyclopaedia as a source of subject defining texts, queries are made to MetaCrawler. This searches several search engines in parallel returning the best 10-20 links. Hesperus retrieves these web pages, and computes their conceptual similarity to the Exemplar Text using a method based on thesaurally determined lexical chains. Initial results show that users prefer the Hesperus page order to MetaCrawler's statistical ordering. The technique consequently shows promise as a way to improve the effectiveness of Web searching.