  Recent literature has demonstrated the potential of multilingual Neural
Machine Translation (mNMT) models. However, the most efficient models are not
well suited to specialized industries. In these cases, internal data is scarce
and expensive to find in all language pairs. Therefore, fine-tuning a mNMT
model on a specialized domain is hard. In this context, we decided to focus on
a new task: Domain Adaptation of a pre-trained mNMT model on a single pair of
language while trying to maintain model quality on generic domain data for all
language pairs. The risk of loss on generic domain and on other pairs is high.
This task is key for mNMT model adoption in the industry and is at the border
of many others. We propose a fine-tuning procedure for the generic mNMT that
combines embeddings freezing and adversarial loss. Our experiments demonstrated
that the procedure improves performances on specialized data with a minimal
loss in initial performances on generic domain for all languages pairs,
compared to a naive standard approach (+10.0 BLEU score on specialized data,
-0.01 to -0.5 BLEU on WMT and Tatoeba datasets on the other pairs with M2M100).
