  Training language models to learn from human instructions for zero-shot
cross-task generalization has attracted much attention in NLP communities.
Recently, instruction tuning (IT), which fine-tunes a pre-trained language
model on a massive collection of tasks described via human-craft instructions,
has been shown effective in instruction learning for unseen tasks. However, IT
relies on a large amount of human-annotated samples, which restricts its
generalization. Unlike labeled data, unlabeled data are often massive and cheap
to obtain. In this work, we study how IT can be improved with unlabeled data.
We first empirically explore the IT performance trends versus the number of
labeled data, instructions, and training tasks. We find it critical to enlarge
the number of training instructions, and the instructions can be underutilized
due to the scarcity of labeled data. Then, we propose Unlabeled Data Augmented
Instruction Tuning (UDIT) to take better advantage of the instructions during
IT by constructing pseudo-labeled data from unlabeled plain texts. We conduct
extensive experiments to show UDIT's effectiveness in various scenarios of
tasks and datasets. We also comprehensively analyze the key factors of UDIT to
investigate how to better improve IT with unlabeled data. The code is publicly
available at https://github.com/thu-coai/UDIT.
