  Training data is the key ingredient for deep learning approaches, but
difficult to obtain for the specialized domains often encountered in robotics.
We describe a synthesis pipeline capable of producing training data for
cluttered scene perception tasks such as semantic segmentation, object
detection, and correspondence or pose estimation. Our approach arranges object
meshes in physically realistic, dense scenes using physics simulation. The
arranged scenes are rendered using high-quality rasterization with randomized
appearance and material parameters. Noise and other transformations introduced
by the camera sensors are simulated. Our pipeline can be run online during
training of a deep neural network, yielding applications in life-long learning
and in iterative render-and-compare approaches. We demonstrate the usability by
learning semantic segmentation on the challenging YCB-Video dataset without
actually using any training frames, where our method achieves performance
comparable to a conventionally trained model. Additionally, we show successful
application in a real-world regrasping system.
