  Lexical ambiguity is a challenging and pervasive problem in machine
translation (\mt). We introduce a simple and scalable approach to resolve
translation ambiguity by incorporating a small amount of extra-sentential
context in neural \mt. Our approach requires no sense annotation and no change
to standard model architectures. Since actual document context is not available
for the vast majority of \mt training data, we collect related sentences for
each input to construct pseudo-documents. Salient words from pseudo-documents
are then encoded as a prefix to each source sentence to condition the
generation of the translation. To evaluate, we release \docmucow, a challenge
set for translation disambiguation based on the English-German \mucow
\cite{raganato-etal-2020-evaluation} augmented with document IDs. Extensive
experiments show that our method translates ambiguous source words better than
strong sentence-level baselines and comparable document-level baselines while
reducing training costs.
