  Using synthetic data for training deep neural networks for robotic
manipulation holds the promise of an almost unlimited amount of pre-labeled
training data, generated safely out of harm's way. One of the key challenges of
synthetic data, to date, has been to bridge the so-called reality gap, so that
networks trained on synthetic data operate correctly when exposed to real-world
data. We explore the reality gap in the context of 6-DoF pose estimation of
known objects from a single RGB image. We show that for this problem the
reality gap can be successfully spanned by a simple combination of domain
randomized and photorealistic data. Using synthetic data generated in this
manner, we introduce a one-shot deep neural network that is able to perform
competitively against a state-of-the-art network trained on a combination of
real and synthetic data. To our knowledge, this is the first deep network
trained only on synthetic data that is able to achieve state-of-the-art
performance on 6-DoF object pose estimation. Our network also generalizes
better to novel environments including extreme lighting conditions, for which
we show qualitative results. Using this network we demonstrate a real-time
system estimating object poses with sufficient accuracy for real-world semantic
grasping of known household objects in clutter by a real robot.
