Conversational agents are exploding in popularity. However, much work remains in the area of non goal-oriented conversations, despite significant growth in research interest over recent years. To advance the state of the art in conversational AI, Amazon launched the Alexa Prize, a 2.5-million dollar university competition where sixteen selected university teams built conversational agents to de...

In this paper we provide an introduction to the field of Bibliometrics. In particular, first we briefly describe its beginning and its evolution; we mention the main research fora as well. Further we categorize metrics according to their entity scope: metrics for journals, conferences and authors. Several rankings have appeared based on such metrics. It is argued that these metrics and rankings...

We introduce and compare three approaches to calculate structureand content-based performance metrics for user-based evaluation of math audio rendering systems: Syntax Tree alignment, Baseline Structure Tree alignment, and MathML Tree Edit Distance. While the first two require “manual” tree transformation and alignment of the mathematical expressions, the third obtains the metrics without human...

String-based metrics of automatic machine translation (MT) evaluation are widely applied in MT research. Meanwhile, some linguistic motivated metrics have been suggested to improve the string-based metrics in sentencelevel evaluation. In this work, we attempt to change their original calculation units (granularities) of string-based metrics to generate new features. We then propose a powerful s...

A number of approaches to Automatic MT Evaluation based on deep linguistic knowledge have been suggested. However, n-gram based metrics are still today the dominant approach. The main reason is that the advantages of employing deeper linguistic information have not been clarified yet. In this work, we propose a novel approach for meta-evaluation of MT evaluation metrics, since correlation coffi...

Metrics and measures for physical phenomena are precisely defined and widely accepted. However, metrics and measures for intelligent systems are as yet vaguely defined and controversial. Even the definition of intelligence is not widely agreed upon. A number of metrics and measures have been developed for measuring human performance in scholastic aptitude, athletic ability, and task performance...

the enormous growth of the world wide web in recent years has made it necessary to perform resource discovery efficiently. for a crawler it is not an simple task to download the domain specific web pages. this unfocused approach often shows undesired results. therefore, several new ideas have been proposed, among them a key technique is focused crawling which is able to crawl particular topical...

There are many unknowns regarding the characteristics and dynamics of human-AI teams, including a lack understanding how certain human-human teaming concepts may or not apply to teams this composition affects team performance. This article outlines an experimental research study that investigates essential aspects such as performance, situation awareness, perceived cognition in various mixed (h...