  The softmax layer in neural machine translation is designed to model the
distribution over mutually exclusive tokens. Machine translation, however, is
intrinsically uncertain: the same source sentence can have multiple
semantically equivalent translations. Therefore, we propose to replace the
softmax activation with a multi-label classification layer that can model
ambiguity more effectively. We call our loss function Single-label Contrastive
Objective for Non-Exclusive Sequences (SCONES). We show that the multi-label
output layer can still be trained on single reference training data using the
SCONES loss function. SCONES yields consistent BLEU score gains across six
translation directions, particularly for medium-resource language pairs and
small beam sizes. By using smaller beam sizes we can speed up inference by a
factor of 3.9x and still match or improve the BLEU score obtained using
softmax. Furthermore, we demonstrate that SCONES can be used to train NMT
models that assign the highest probability to adequate translations, thus
mitigating the "beam search curse". Additional experiments on synthetic
language pairs with varying levels of uncertainty suggest that the improvements
from SCONES can be attributed to better handling of ambiguity.
