  One of the major challenges of machine translation (MT) is ambiguity, which
can in some cases be resolved by accompanying context such as images. However,
recent work in multimodal MT (MMT) has shown that obtaining improvements from
images is challenging, limited not only by the difficulty of building effective
cross-modal representations, but also by the lack of specific evaluation and
training data. We present a new MMT approach based on a strong text-only MT
model, which uses neural adapters, a novel guided self-attention mechanism and
which is jointly trained on both visually-conditioned masking and MMT. We also
introduce CoMMuTE, a Contrastive Multilingual Multimodal Translation Evaluation
set of ambiguous sentences and their possible translations, accompanied by
disambiguating images corresponding to each translation. Our approach obtains
competitive results compared to strong text-only models on standard
English-to-French, English-to-German and English-to-Czech benchmarks and
outperforms baselines and state-of-the-art MMT systems by a large margin on our
contrastive test set. Our code and CoMMuTE are freely available.
