  We introduce a novel learning method for 3D pose estimation from color
images. While acquiring annotations for color images is a difficult task, our
approach circumvents this problem by learning a mapping from paired color and
depth images captured with an RGB-D camera. We jointly learn the pose from
synthetic depth images that are easy to generate, and learn to align these
synthetic depth images with the real depth images. We show our approach for the
task of 3D hand pose estimation and 3D object pose estimation, both from color
images only. Our method achieves performances comparable to state-of-the-art
methods on popular benchmark datasets, without requiring any annotations for
the color images.
