  Emergent multi-agent communication protocols are very different from natural
language and not easily interpretable by humans. We find that agents that were
initially pretrained to produce natural language can also experience
detrimental language drift: when a non-linguistic reward is used in a
goal-based task, e.g. some scalar success metric, the communication protocol
may easily and radically diverge from natural language. We recast translation
as a multi-agent communication game and examine auxiliary training constraints
for their effectiveness in mitigating language drift. We show that a
combination of syntactic (language model likelihood) and semantic (visual
grounding) constraints gives the best communication performance, allowing
pre-trained agents to retain English syntax while learning to accurately convey
the intended meaning.
