  k-nearest-neighbor machine translation (NN-MT), proposed by Khandelwal et al.
(2021), has achieved many state-of-the-art results in machine translation
tasks. Although effective, NN-MT requires conducting NN searches through the
large datastore for each decoding step during inference, prohibitively
increasing the decoding cost and thus leading to the difficulty for the
deployment in real-world applications. In this paper, we propose to move the
time-consuming NN search forward to the preprocessing phase, and then introduce
Nearest Neighbor Knowledge Distillation (NN-KD) that trains the base NMT model
to directly learn the knowledge of NN. Distilling knowledge retrieved by NN can
encourage the NMT model to take more reasonable target tokens into
consideration, thus addressing the overcorrection problem. Extensive
experimental results show that, the proposed method achieves consistent
improvement over the state-of-the-art baselines including NN-MT, while
maintaining the same training and decoding speed as the standard NMT model.
