  Many deep learning tasks require annotations that are too time consuming for
human operators, resulting in small dataset sizes. This is especially true for
dense regression problems such as crowd counting which requires the location of
every person in the image to be annotated. Techniques such as data augmentation
and synthetic data generation based on simulations can help in such cases. In
this paper, we introduce PromptMix, a method for artificially boosting the size
of existing datasets, that can be used to improve the performance of
lightweight networks. First, synthetic images are generated in an end-to-end
data-driven manner, where text prompts are extracted from existing datasets via
an image captioning deep network, and subsequently introduced to text-to-image
diffusion models. The generated images are then annotated using one or more
high-performing deep networks, and mixed with the real dataset for training the
lightweight network. By extensive experiments on five datasets and two tasks,
we show that PromptMix can significantly increase the performance of
lightweight networks by up to 26%.
