  We propose a simple method to generate multilingual question and answer pairs
on a large scale through the use of a single generative model. These synthetic
samples can be used to improve the zero-shot performance of multilingual QA
models on target languages. Our proposed multi-task training of the generative
model only requires the labeled training samples in English, thus removing the
need for such samples in the target languages, making it applicable to far more
languages than those with labeled data. Human evaluations indicate the majority
of such samples are grammatically correct and sensible. Experimental results
show our proposed approach can achieve large gains on the XQuAD dataset,
reducing the gap between zero-shot and supervised performance of smaller QA
models on various languages.
