  We propose a new paradigm to automatically generate training data with
accurate labels at scale using the text-toimage synthesis frameworks (e.g.,
DALL-E, Stable Diffusion, etc.). The proposed approach decouples training data
generation into foreground object mask generation and background (context)
image generation. For foreground object mask generation, we use a simple
textual template with object class name as input to DALL-E to generate a
diverse set of foreground images. A foreground-background segmentation
algorithm is then used to generate foreground object masks. Next, in order to
generate context images, first a language description of the context is
generated by applying an image captioning method on a small set of images
representing the context. These language descriptions are then used to generate
diverse sets of context images using the DALL-E framework. These are then
composited with object masks generated in the first step to provide an
augmented training set for a classifier. We demonstrate the advantages of our
approach on four object detection datasets including on Pascal VOC and COCO
object detection tasks. Furthermore, we also highlight the compositional nature
of our data generation approach on out-of-distribution and zero-shot data
generation scenarios.
