Tarantula - A Scalable and Extensible Web Spider
نویسندگان
چکیده
Web crawlers today suffer from poor navigation techniques which reduce their scalability while crawling the World Wide Web (WWW). In this paper we present a web crawler named Tarantula that is scalable and fully configurable. The work on Tarantula project was started with the aim of making a simple, elegant and yet an efficient Web Crawler offering better crawling strategies while walking through the WWW. This paper also presents a comparison with the Heritrix (Mohr et al.) crawler. The structure of the crawler facilitates new navigation techniques which can be used with existing techniques to give improved crawl results. Tarantula has a pluggable, extensible architecture that further facilitates customization by the user.
منابع مشابه
The Aromatic Head Group of Spider Toxin Polyamines Influences Toxicity to Cancer Cells
Spider venoms constitute incredibly diverse libraries of compounds, many of which are involved in prey capture and defence. Polyamines are often prevalent in the venom and target ionotropic glutamate receptors. Here we show that a novel spider polyamine, PA366, containing a hydroxyphenyl-based structure is present in the venom of several species of tarantula, and has selective toxicity against ...
متن کامل[Dancing manias. Between culture and medicine].
Dancing mania is a clinical and cultural phenomenon which occurred in Western Europe between 13th and 18th centuries. The term dancing mania is derived from the Greek words choros, a dance, and mania, a madness. An Italian variant was known as tarantism as victims were believed to have been bitten by tarantula spider. Although symptoms of dancing manias were well documented in contemporary writ...
متن کاملVisual odometry in the wolf spider Lycosa tarantula (Araneae: Lycosidae).
The wolf spider Lycosa tarantula homes using path integration. The angular component of the displacement is measured using a polarized-light compass associated with the functioning of the anterior median eyes. However, how L. tarantula estimates the linear component of the displacement was not known prior to this investigation. The ability of L. tarantula to gauge the distance walked after bein...
متن کاملSymptom in search of a toxin: muscle spasms following bites by Old World tarantula spiders (Lampropelma nigerrimum, Pterinochilus murinus, Poecilotheria regalis) with review.
BACKGROUND Tarantula spiders are widely kept and bred in captivity by both adults and children. Their bites are generally considered harmless. AIM To explore the effects of envenoming by Old World tarantulas. DESIGN AND METHODS Clinical studies and review of conventional literature and hobbyist web sites. RESULTS Two men bitten on their index fingers by pet Old World tarantula spiders, La...
متن کاملVenom Profiling of a Population of the Theraphosid Spider Phlogius crassipes Reveals Continuous Ontogenetic Changes from Juveniles through Adulthood
Theraphosid spiders (tarantulas) are venomous arthropods found in most tropical and subtropical regions of the world. Tarantula venoms are a complex cocktail of toxins with potential use as pharmacological tools, drugs and bioinsecticides. Although numerous toxins have been isolated from tarantula venoms, little research has been carried out on the venom of Australian tarantulas. We therefore i...
متن کامل