  The power of DNNs relies heavily on the quantity and quality of training
data. However, collecting and annotating data on a large scale is often
expensive and time-consuming. To address this issue, we explore a new task,
termed dataset expansion, aimed at expanding a ready-to-use small dataset by
automatically creating new labeled samples. To this end, we present a Guided
Imagination Framework (GIF) that leverages cutting-edge generative models like
DALL-E2 and Stable Diffusion (SD) to "imagine" and create informative new data
from the input seed data. Specifically, GIF conducts data imagination by
optimizing the latent features of the seed data in the semantically meaningful
space of the prior model, resulting in the creation of photo-realistic images
with new content. To guide the imagination towards creating informative samples
for model training, we introduce two key criteria, i.e., class-maintained
information boosting and sample diversity promotion. These criteria are
verified to be essential for effective dataset expansion: GIF-SD obtains 13.5%
higher model accuracy on natural image datasets than unguided expansion with
SD. With these essential criteria, GIF successfully expands small datasets in
various scenarios, boosting model accuracy by 36.9% on average over six natural
image datasets and by 13.5% on average over three medical datasets. The source
code is available at https://github.com/Vanint/DatasetExpansion.
