  Data augmentation is an essential technique in improving the generalization
of deep neural networks. The majority of existing image-domain augmentations
either rely on geometric and structural transformations, or apply different
kinds of photometric distortions. In this paper, we propose an effective
technique for image augmentation by injecting contextually meaningful knowledge
into the scenes. Our method of semantically meaningful image augmentation for
object detection via language grounding, SemAug, starts by calculating
semantically appropriate new objects that can be placed into relevant locations
in the image (the what and where problems). Then it embeds these objects into
their relevant target locations, thereby promoting diversity of object instance
distribution. Our method allows for introducing new object instances and
categories that may not even exist in the training set. Furthermore, it does
not require the additional overhead of training a context network, so it can be
easily added to existing architectures. Our comprehensive set of evaluations
showed that the proposed method is very effective in improving the
generalization, while the overhead is negligible. In particular, for a wide
range of model architectures, our method achieved ~2-4% and ~1-2% mAP
improvements for the task of object detection on the Pascal VOC and COCO
datasets, respectively.
