  Minimum Bayesian Risk Decoding (MBR) emerges as a promising decoding
algorithm in Neural Machine Translation. However, MBR performs poorly with
label smoothing, which is surprising as label smoothing provides decent
improvement with beam search and improves generality in various tasks. In this
work, we show that the issue arises from the un-consistency of label smoothing
on the token-level and sequence-level distributions. We demonstrate that even
though label smoothing only causes a slight change in the token-level, the
sequence-level distribution is highly skewed. We coin the issue
\emph{autoregressive over-smoothness}. To address this issue, we propose a
simple and effective method, Distributional Cooling MBR (DC-MBR), which
manipulates the entropy of output distributions by tuning down the Softmax
temperature. We theoretically prove the equivalence between pre-tuning label
smoothing factor and distributional cooling. Extensive experiments on NMT
benchmarks validate that distributional cooling improves MBR in various
settings.
