RRLUFF: Ranking function based on Reinforcement Learning using User Feedback and Web Document Features

Principal aim of a search engine is to provide the sorted results according to user’s requirements. To achieve this aim, it employs ranking methods to rank the web documents based on their significance and relevance to user query. The novelty of this paper is to provide user feedback-based ranking algorithm using reinforcement learning. The proposed algorithm is called RRLUFF, in which the ranking system is considered as the agent of the learning system and the selection of documents is displayed to the user as the agent's action. Reinforcement signal in this system is calculated based on user's click on the documents. Action-values in the RRLUFF algorithm are calculated for each feature of the document-query pair. In RRLUFF method, each feature is scored based on the number of the documents related to the query and their position in the ranked list of that feature. For learning, documents are sorted according to modified scores for the next query. Then, according to the position of a document in the ranking list, some documents are selected based on the random distribution of their scores to display to the user. OHSUMED and DOTIR benchmark datasets are used to evaluate the proposed method. The evaluation results indicate that the proposed method is more effective than the related methods in terms of P@n, NDCG@n, MAP, and NWN.