In the present work we study semi-automatic evaluation techniques of machine translation (MT) systems. These techniques are based on a comparison of the MT system’s output to human translations of the same text. Various metrics were proposed in the recent years, ranging from metrics using only a unigram comparison to metrics that try to take advantage of additional syntactic or semantic informa...

conclusions by measuring image quality metrics, this study showed that dtd and ctd filters with the canny edge detection respectively, are better than srad filter with the canny detection for speckle suppression and details preservation in both arteries in the ultrasound images. background ultrasonic evaluation of intima-media thickness (imt) is an early marker of assessing the development of a...

Evaluation metrics for image captioning face two challenges. Firstly, commonly used metrics such as CIDEr, METEOR, ROUGE and BLEU often do not correlate well with human judgments. Secondly, each metric has well known blind spots to pathological caption constructions, and rulebased metrics lack provisions to repair such blind spots once identified. For example, the newly proposed SPICE correlate...

The Metrics for Human-Robot Interaction 2008 workshop at the 3rd ACM/IEEE International Conference on Human-Robot Interaction was initiated and organized to further discussion and community progress towards metrics for human-robot interaction (HRI). This report contains the papers presented at the workshop, background information on the workshop itself, and future directions underway within the...

Evaluation of segment-level machine translation metrics is currently hampered by: (1) low inter-annotator agreement levels in human assessments; (2) lack of an effective mechanism for evaluation of translations of equal quality; and (3) lack of methods of significance testing improvements over a baseline. In this paper, we provide solutions to each of these challenges and outline a new human ev...

Key factors that a hospital finance leader should focus on when considering a potential co-management agreement with physicians, in which the physicians are compensated at fair market value, include: Fee structure of the agreement. The quality metrics that will be used. Benchmarking to set appropriate targets for metrics. Historical performance against the metrics. Legal guidelines regarding su...

xiii

Available online xxxx

Current methods for automatically evaluating grammatical error correction (GEC) systems rely on gold-standard references. However, these methods suffer from penalizing grammatical edits that are correct but not in the gold standard. We show that reference-less grammaticality metrics correlate very strongly with human judgments and are competitive with the leading reference-based evaluation metr...