  Noisy channel models have been especially effective in neural machine
translation (NMT). However, recent approaches like "beam search and rerank"
(BSR) incur significant computation overhead during inference, making
real-world application infeasible. We aim to study if it is possible to build
an amortized noisy channel NMT model such that when we do greedy decoding
during inference, the translation accuracy matches that of BSR in terms of
reward (based on the source-to-target log probability and the target-to-source
log probability) and quality (based on BLEU and BLEURT). We attempt three
approaches to train the new model: knowledge distillation, one-step-deviation
imitation learning, and Q learning. The first approach obtains the noisy
channel signal from a pseudo-corpus, and the latter two approaches aim to
optimize toward a noisy-channel MT reward directly. For all three approaches,
the generated translations fail to achieve rewards comparable to BSR, but the
translation quality approximated by BLEU and BLEURT is similar to the quality
of BSR-produced translations. Additionally, all three approaches speed up
inference by 1-2 orders of magnitude.
