Traditional automatic evaluation measures for natural language generation (NLG) use costly human-authored references to estimate the quality of a system output. In this paper, we propose a referenceless quality estimation (QE) approach based on recurrent neural networks, which predicts a quality score for a NLG system output by comparing it to the source meaning representation only. Our method ...

The choice of an input representation for a neural network can have a profound impact on its accuracy in classifying novel instances. However, neural networks are typically computationally expensive to train, making it diicult to test large numbers of alternative representations. This paper introduces fast quality measures for neural network representations, allowing one to quickly and accurate...

We propose a novel method to estimate the quality of coded speech signals. The joint probability distribution of the subjective mean opinion score (MOS) and perceptual distortion feature variables is modelled using a Gaussian mixture density. The feature variables are sifted from a large pool of candidate features using statistical data mining techniques. We study what combinations of features ...

Automatic post-editing (APE) is a challenging task on WMT evaluation campaign. We find that only a small number of edit operations are required for most machine translation outputs, through analysis of the training set of WMT17 APE en-de task. Based on this statistics analysis, two neural postediting (NPE) models are trained depended on the edit numbers: single edit and minor edits. The improve...

In this paper we introduce a number of new features for quality estimation in machine translation that were developed for the WMT 2012 quality estimation shared task. We find that very simple features such as indicators of certain characters are able to outperform complex features that aim to model the connection between two languages.

This paper describes the machine learning algorithm and the features used by LIMSI for the Quality Estimation Shared Task. Our submission mainly aims at evaluating the usefulness for quality estimation of ngram posterior probabilities that quantify the probability for a given n-gram to be part of the system output.

Wireless sensor nodes deployed in RF industrial environment for monitoring machineries and environment for controlling and maintenance purpose is challenging due to exchange of huge critical data via degraded link. The quality of link in industrial environment is highly influenced by metallic infrastructure, multipath distortion, EM noise, wide tolerance range of device and coexisting technolog...

In order to cooperate, smart objects need to communicate using wireless communication. Wireless communication, however, is inherently unreliable. Towards this end, the sensor networking community has developed a number of metrics for link quality estimation. Most of these metrics, however, are designed for static sensor networks whereas cooperating objects in many cases are mobile. In this posi...

Post-Editing of Machine Translation (MT) has become a reality in professional translation workflows. In order to optimize the management of projects that use post-editing and avoid underpayments and mistrust from professional translators, effective tools to assess the quality of Machine Translation (MT) systems need to be put in place. One field of study that could address this problem is Machi...

This paper presents a novel approach for parallel data generation using machine translation and quality estimation. Our study focuses on pivot-based machine translation from English to Croatian through Slovene. We generate an English–Croatian version of the Europarl parallel corpus based on the English–Slovene Europarl corpus and the Apertium rule-based translation system for Slovene–Croatian. ...