This study aims at investigating procedures of semantic and linguistic extraction keywords from metadata documents indexed in the Institutional Repository Unesp. For that purpose, a web crawler was developed, collected 325.181 authors, all fields knowledge, February 28th, 2013 to November 10th, 2021. The preparation collection, analysis environment used Python programming language, composed thr...

In this paper we present a design and implementation of a scalable, distributed web-crawler. The motivation for design of such a system to effectively distribute crawling tasks to different machined in a peer-peer distributed network. Such architecture will lead to scalability and help tame the exponential growth or crawl space in the World Wide Web. With experiments on the implementation of th...

This paper advocated the use of ontology-supported website models to provide a semantic level solution for an information agent so that it can provide fast, precise, and stable query results. Based on the technique, a focused crawler, namely, OntoCrawler, was developed, which can benefit both user requests and domain semantics. Equipped with this technique, we have developed an ontology-support...

JavaScript Client-side hidden web pages (CSHW) contain dynamic material created as a result of specific user activities. The number of CSHW websites is increasing. Crawling the so-called Hidden Web is challenging, particularly when JavaScript CSHW from an external website is seamlessly included as part of the web pages. We have developed a prototype web crawler that efficiently extracts content...

Web crawler uncontrolled widespread has led to undesired situations of server overload and contents misuse. Most programs still have legitimate and useful goals, but standard detection heuristics have not evolved along with Web crawling technology and are now unable to identify most of today’s programs. In this paper, we propose an integrated approach to the problem that ensures the generation ...

We summarize the economic importance, biology, and management of soft scales, focusing on pests of agricultural, horticultural, and silvicultural crops in outdoor production systems and urban landscapes. We also provide summaries on voltinism, crawler emergence timing, and predictive models for crawler emergence to assist in developing soft scale management programs. Phloem-feeding soft scale p...

This paper devotes to discovering the high-quality users from Sina microblog (Weibo) which is the most popular microblog site in China. First, the Trust Transfer Model (TTM) is introduced as a theoretical background to make sure that users are trustworthy and high-quality. Then, a Breadth First Search (BFS) crawler based on TTM is implemented to capture users’ profile data via Weibo APIs. There...

Nowadays, there is a trend to create resource-consuming applications without building heavy computer centers, but to use resources on computer systems distributed over the internet. Grid middleware is a framework to access these resources. The concern of this paper is the evaluation of a specific grid middleware, namely Globus Toolkit, for data-intensive applications. As a test case, we have de...