  Despite the progress in machine translation quality estimation and evaluation
in the last years, decoding in neural machine translation (NMT) is mostly
oblivious to this and centers around finding the most probable translation
according to the model (MAP decoding), approximated with beam search. In this
paper, we bring together these two lines of research and propose quality-aware
decoding for NMT, by leveraging recent breakthroughs in reference-free and
reference-based MT evaluation through various inference methods like $N$-best
reranking and minimum Bayes risk decoding. We perform an extensive comparison
of various possible candidate generation and ranking methods across four
datasets and two model classes and find that quality-aware decoding
consistently outperforms MAP-based decoding according both to state-of-the-art
automatic metrics (COMET and BLEURT) and to human assessments. Our code is
available at https://github.com/deep-spin/qaware-decode.
