  Cross-lingual transfer is a promising technique for utilizing data in a
source language to improve performance in a target language. However, current
techniques often require an external translation system or suffer from
suboptimal performance due to over-reliance on cross-lingual generalization of
multi-lingual pretrained language models. In this study, we propose a simple
yet effective method called Self-Translate-Train. It leverages the translation
capability of a large language model to generate synthetic training data in the
target language and fine-tunes the model with its own generated data. We
evaluate the proposed method on a wide range of tasks and show substantial
performance gains across several non-English languages.
