Information retrieval on Internet using meta-search engines: A review

Introduction Though automatic information retrieval (IR) existed before World Wide Web (WWW), post-Internet era has made it indispensable. IR is sub field of computer science concerned with presenting relevant information, gathered from online information sources to users in response to search queries. Various types of IR tools have been created, solely to search information on Internet. Apart from heavily used search engines (SEs), other useful tools are deep-web search portals, webdirectories and meta-search engines (MSEs). Among various IR tools available, most people use SEs to locate web pages. Using SEs, an index is searched rather than entire Web. Index is created and maintained through ongoing automated web searching by programs commonly known as spiders. Web-directories are databases of web sites compiled and maintained by humans. Since web-directory content is hand picked by humans, results have high accuracy but a typical webdirectory’s index size will be only a fraction of that of a SE and content can easily become outdated. OpenDirectory-Project (ODP) is biggest web-directory available today. Deep-Web or Invisible web consists of publicly accessible pages with information in databases such as catalogues and references on WWW that are not indexed by SEs. Much research has emerged in recent years on information in invisible web sites. A MSE (multi-threaded SE) such as Dogpile, SavvySearch, Metacrawler, Profusion and Inquirus, is a search tool that sends a query simultaneously to several SEs and consolidates all results, thereby saving time. This paper reviews working of MSEs and their significance in IR on Internet.