  Understanding representation transfer in multilingual neural machine
translation can reveal the representational issue causing the zero-shot
translation deficiency. In this work, we introduce the identity pair, a
sentence translated into itself, to address the lack of the base measure in
multilingual investigations, as the identity pair represents the optimal state
of representation among any language transfers. In our analysis, we demonstrate
that the encoder transfers the source language to the representational subspace
of the target language instead of the language-agnostic state. Thus, the
zero-shot translation deficiency arises because representations are entangled
with other languages and are not transferred effectively to the target
language. Based on our findings, we propose two methods: 1) low-rank
language-specific embedding at the encoder, and 2) language-specific
contrastive learning of the representation at the decoder. The experimental
results on Europarl-15, TED-19, and OPUS-100 datasets show that our methods
substantially enhance the performance of zero-shot translations by improving
language transfer capacity, thereby providing practical evidence to support our
conclusions.
