  We present a novel approach to tackle domain adaptation between synthetic and
real data. Instead, of employing "blind" domain randomization, i.e., augmenting
synthetic renderings with random backgrounds or changing illumination and
colorization, we leverage the task network as its own adversarial guide toward
useful augmentations that maximize the uncertainty of the output. To this end,
we design a min-max optimization scheme where a given task competes against a
special deception network to minimize the task error subject to the specific
constraints enforced by the deceiver. The deception network samples from a
family of differentiable pixel-level perturbations and exploits the task
architecture to find the most destructive augmentations. Unlike GAN-based
approaches that require unlabeled data from the target domain, our method
achieves robust mappings that scale well to multiple target distributions from
source data alone. We apply our framework to the tasks of digit recognition on
enhanced MNIST variants, classification and object pose estimation on the
Cropped LineMOD dataset as well as semantic segmentation on the Cityscapes
dataset and compare it to a number of domain adaptation approaches, thereby
demonstrating similar results with superior generalization capabilities.
