  LLMs have become a go-to solution not just for text generation, but also for
natural language understanding (NLU) tasks. Acquiring extensive knowledge
through language modeling on web-scale corpora, they excel on English NLU, yet
struggle to extend their NLU capabilities to underrepresented languages. In
contrast, machine translation models (MT) produce excellent multilingual
representations, resulting in strong translation performance even for
low-resource languages. MT encoders, however, lack the knowledge necessary for
comprehensive NLU that LLMs obtain through language modeling training on
immense corpora. In this work, we get the best both worlds by integrating MT
encoders directly into LLM backbones via sample-efficient self-distillation.
The resulting MT-LLMs preserve the inherent multilingual representational
alignment from the MT encoder, allowing lower-resource languages to tap into
the rich knowledge embedded in English-centric LLMs. Merging the MT encoder and
LLM in a single model, we mitigate the propagation of translation errors and
inference overhead of MT decoding inherent to discrete translation-based
cross-lingual transfer (e.g., translate-test). Evaluation spanning three
prominent NLU tasks and 127 predominantly low-resource languages renders
MT-LLMs highly effective in cross-lingual transfer. MT-LLMs substantially and
consistently outperform translate-test based on the same MT model, showing that
we truly unlock multilingual language understanding for LLMs.
