  Natural language generation (NLG) benchmarks provide an important avenue to
measure progress and develop better NLG systems. Unfortunately, the lack of
publicly available NLG benchmarks for low-resource languages poses a
challenging barrier for building NLG systems that work well for languages with
limited amounts of data. Here we introduce IndoNLG, the first benchmark to
measure natural language generation (NLG) progress in three low-resource -- yet
widely spoken -- languages of Indonesia: Indonesian, Javanese, and Sundanese.
Altogether, these languages are spoken by more than 100 million native
speakers, and hence constitute an important use case of NLG systems today.
Concretely, IndoNLG covers six tasks: summarization, question answering,
chit-chat, and three different pairs of machine translation (MT) tasks. We
collate a clean pretraining corpus of Indonesian, Sundanese, and Javanese
datasets, Indo4B-Plus, which is used to pretrain our models: IndoBART and
IndoGPT. We show that IndoBART and IndoGPT achieve competitive performance on
all tasks -- despite using only one-fifth the parameters of a larger
multilingual model, mBART-LARGE (Liu et al., 2020). This finding emphasizes the
importance of pretraining on closely related, local languages to achieve more
efficient learning and faster inference for very low-resource languages like
Javanese and Sundanese.
