A Visual Analytics Tool for Experimental Evaluation
نویسندگان
چکیده
Information Retrieval is a research field strongly rooted in experimentation. Indeed, measuring is a key to scientific progress. Multilingual and multimedia information access systems, such as search engines, are increasingly complex: they need to satisfy diverse user needs and support challenging tasks. It is therefore fundamental to provide automated tools to examine system behaviour, both visually and analytically. This paper provides an analytical model for examining performances of IR systems, based on the discounted cumulative gain family of metrics, and visualization for interacting and exploring the performances of the system under examination.
منابع مشابه
What-If Analysis: A Visual Analytics Approach to Information Retrieval Evaluation
This paper focuses on the innovative visual analytics approach realized by the Visual Analytics Tool for Experimental Evaluation (VATE2) system, which eases and makes more effective the experimental evaluation process by introducing the what-if analysis. The what-if analysis is aimed at estimating the possible effects of a modification to an Information Retrieval (IR) system, in order to select...
متن کاملVisual Analytics for Information Retrieval Evaluation (VAIRË 2015)
Measuring is a key to scientific progress. This is particularly true for research concerning complex systems, whether natural or human-built. The tutorial introduced basic and intermediate concepts about lab-based evaluation of information retrieval systems, its pitfalls, and shortcomings and it complemented them with a recent and innovative angle to evaluation: the application of methodologies...
متن کاملUsing Sentence Compression to Develop Visual Analytics for Student Responses to Short Answer Questions
In this paper, we report on early research to visualize and summarize student responses to short answer questions. Recently published, graph-based multi-sentence compression algorithms have been successfully applied to summarize opinions – a domain area with many similarities to short answer responses. Initial investigations reveal that visual analytics for short answer questions can be derived...
متن کاملComparative Evaluation of Two Interface Tools in Performing Visual Analytics Tasks
In this paper, we performed a comparative evaluation to show the effectiveness of two interface tools, one a floating text-based menu (Floating-Menu) and the other a more interactive iconic tool (Interactive-Icon). We evaluated the use and human performance of both tools within one highly interactive visual analytics system. During task performance we tracked completion times, task errors, and ...
متن کاملImproving Information Retrieval Evaluation via Markovian User Models and Visual Analytics
To address the challenge of adapting experimental evaluation to the constantly evolving user tasks and needs, we develop a new family of Markovian Information Retrieval (IR) evaluation measures, called Markov Precision (MP), where the interaction between the user and the ranked result list is modelled via Markov chains, and which will be able to explicitly link lab-style and on-line evaluation ...
متن کامل