  Language coverage bias, which indicates the content-dependent differences
between sentence pairs originating from the source and target languages, is
important for neural machine translation (NMT) because the target-original
training data is not well exploited in current practice. By carefully designing
experiments, we provide comprehensive analyses of the language coverage bias in
the training data, and find that using only the source-original data achieves
comparable performance with using full training data. Based on these
observations, we further propose two simple and effective approaches to
alleviate the language coverage bias problem through explicitly distinguishing
between the source- and target-original training data, which consistently
improve the performance over strong baselines on six WMT20 translation tasks.
Complementary to the translationese effect, language coverage bias provides
another explanation for the performance drop caused by back-translation. We
also apply our approach to both back- and forward-translation and find that
mitigating the language coverage bias can improve the performance of both the
two representative data augmentation methods and their tagged variants.
