  Advancements in Large Language Models (LLMs) have significantly enhanced
instruction-following capabilities. However, most Instruction Fine-Tuning (IFT)
datasets are predominantly in English, limiting model performance in other
languages. Traditional methods for creating multilingual IFT datasets such as
translating existing English IFT datasets or converting existing NLP datasets
into IFT datasets by templating, struggle to capture linguistic nuances and
ensure prompt (instruction) diversity. To address this issue, we propose a
novel method for collecting multilingual IFT datasets that preserves linguistic
naturalness and ensures prompt diversity. This approach leverages
English-focused LLMs, monolingual corpora, and a scoring function to create
high-quality, diversified IFT datasets in multiple languages. Experiments
demonstrate that LLMs finetuned using these IFT datasets show notable
improvements in both generative and discriminative tasks, indicating enhanced
language comprehension by LLMs in non-English contexts. Specifically, on the
multilingual summarization task, LLMs using our IFT dataset achieved 17.57% and
15.23% improvements over LLMs fine-tuned with translation-based and
template-based datasets, respectively.
