Rephrasing the Reference for Non-autoregressive Machine Translation

Non-autoregressive neural machine translation (NAT) models suffer from the multi-modality problem that there may exist multiple possible translations of a source sentence, so reference sentence be inappropriate for training when NAT output is closer to other translations. In response this problem, we introduce rephraser provide better target by rephrasing according output. As train based on rather than should fit well with and not deviate too far reference, which can quantified as reward functions optimized reinforcement learning. Experiments major WMT benchmarks baselines show our approach consistently improves quality NAT. Specifically, best variant achieves comparable performance autoregressive Transformer, while being 14.7 times more efficient in inference.