  Instruction tuning benefits from large and diverse datasets, however creating
such datasets involves a high cost of human labeling. While synthetic datasets
generated by large language models (LLMs) have partly solved this issue, they
often contain low-quality data. One effective solution is selectively
annotating unlabelled instructions, especially given the relative ease of
acquiring unlabeled instructions or texts from various sources. However, how to
select unlabelled instructions is not well-explored, especially in the context
of LLMs. Further, traditional data selection methods, relying on input
embedding space density, tend to underestimate instruction sample complexity,
whereas those based on model prediction uncertainty often struggle with
synthetic label quality. Therefore, we introduce SelectLLM, an alternative
framework that leverages the capabilities of LLMs to more effectively select
unlabeled instructions. SelectLLM consists of two key steps: Coreset-based
clustering of unlabelled instructions for diversity and then prompting a LLM to
identify the most beneficial instructions within each cluster. Our experiments
demonstrate that SelectLLM matches or outperforms other state-of-the-art
methods in instruction tuning benchmarks. It exhibits remarkable consistency
across human and synthetic datasets, along with better cross-dataset
generalization, as evidenced by a 10% performance improvement on the Cleaned
Alpaca test set when trained on Dolly data. All code and data are publicly
available (https://github.com/minnesotanlp/select-llm).
