  Nearest Neighbor Machine Translation (kNNMT) is a simple and effective method
of augmenting neural machine translation (NMT) with a token-level nearest
neighbor retrieval mechanism. The effectiveness of kNNMT directly depends on
the quality of retrieved neighbors. However, original kNNMT builds datastores
based on representations from NMT models, which would result in poor retrieval
accuracy when NMT models are not good enough, leading to sub-optimal
translation performance. In this paper, we propose PRED, a framework that
leverages Pre-trained models for Datastores in kNN-MT. Better representations
from pre-trained models allow us to build datastores of better quality. We also
design a novel contrastive alignment objective to mitigate the representation
gap between the NMT model and pre-trained models, enabling the NMT model to
retrieve from better datastores. We conduct extensive experiments on both
bilingual and multilingual translation benchmarks, including WMT17 English
$\leftrightarrow$ Chinese, WMT14 English $\leftrightarrow$ German, IWSLT14
German $\leftrightarrow$ English, and IWSLT14 multilingual datasets. Empirical
results demonstrate the effectiveness of PRED.
