  Instruction tuning is a standard technique employed to align large language
models to end tasks and user preferences after the initial pretraining phase.
Recent research indicates the critical role of data engineering in instruction
tuning -- when appropriately selected, only limited data is necessary to
achieve superior performance. However, we still lack a principled understanding
of what makes good instruction tuning data for alignment, and how we should
select data automatically and effectively. In this work, we delve deeply into
automatic data selection strategies for alignment. We start with controlled
studies to measure data across three dimensions: complexity, quality, and
diversity, along which we examine existing methods and introduce novel
techniques for enhanced data measurement. Subsequently, we propose a simple
strategy to select data samples based on the measurement. We present deita
(short for Data-Efficient Instruction Tuning for Alignment), a series of models
fine-tuned from LLaMA and Mistral models using data samples automatically
selected with our proposed approach. Empirically, deita performs better or on
par with the state-of-the-art open-source alignment models with only 6K SFT
training data samples -- over 10x less than the data used in the baselines.
When further trained with direct preference optimization (DPO),
deita-Mistral-7B + DPO trained with 6K SFT and 10K DPO samples achieve 7.55
MT-Bench and 90.06% AlpacaEval scores. We anticipate this work to provide tools
on automatic data selection, facilitating data-efficient alignment. We release
our models as well as the selected datasets for future researches to
effectively align models more efficiently.
