1 This paper proposes a categorization of reinforcement learning environments based on the optimization of a reinforcement signal over time. Environments are classiied by the simplest agent that can possibly achieve optimal reinforcement. Two parameters, h and , abstractly characterize the complexity of an agent: the ideal (h,)-agent uses the input information provided by the environment and at...

We introduce a new framework for web page ranking—reinforcement ranking—that improves the stability and accuracy of Page Rank while eliminating the need for computing the stationary distribution of random walks. Instead of relying on teleportation to ensure a well defined Markov chain, we develop a reverse-time reinforcement learning framework that determines web page authority based on the sol...

As the Web continues to grow rapidly, focused topic-specific Web crawlers will gain popularity over traditional general-purpose search engines for locating, indexing and keeping up to date information on the Web. This paper presents YAFC (Yet Another Focused Crawler), a neurodynamic programming approach to focused crawling. YAFC combines TD(λ) reinforcement learning with a neural network to lea...

Document filtering is increasingly deployed in Web environments to reduce information overload of users. We formulate online information filtering as a reinforcement learning problem, i.e. TD(0). The goal is to learn user profiles that best represent his information needs and thus maximize the expected value of user relevance feedback. A method is then presented that acquires reinforcement sign...

Deep web refers to the hidden part of the Web that remains unavailable for standard Web crawlers. To obtain content of Deep Web is challenging and has been acknowledged as a significant gap in the coverage of search engines. To this end, the paper proposes a novel deep web crawling framework based on reinforcement learning, in which the crawler is regarded as an agent and deep web database as t...

Today, when searching for information on the World Wide Web, one usually performs a query through a term-based search engine. These engines return, as the query’s result, a list of Web sites whose contents match the query. For broad topic queries, such searches often result in a huge set of retrieved documents, many of which are irrelevant to the user. However, much information is contained in ...

Deep Web is autonomous, independently updating, and its data are always in a state of frequent update. However, the user always hopes to obtain the newest content in the current Web database. Different from previous research, this paper wants to emphasize the importance of updating frequency in the study of Deep Web information acquisition. And, an approach on incremental information acquisitio...

Ranking web pages for presenting the most relevant web pages to user’s queries is one of the main issues in any search engine. In this paper, two new ranking algorithms are offered, using Reinforcement Learning (RL) concepts. RL is a powerful technique of modern artificial intelligence that tunes agent’s parameters, interactively. In the first step, with formulation of ranking as an RL problem,...