  We tackle the problem of learning the geometry of multiple categories of
deformable objects jointly. Recent work has shown that it is possible to learn
a unified dense pose predictor for several categories of related objects.
However, training such models requires to initialize inter-category
correspondences by hand. This is suboptimal and the resulting models fail to
maintain correct correspondences as individual categories are learned. In this
paper, we show that improved correspondences can be learned automatically as a
natural byproduct of learning category-specific dense pose predictors. To do
this, we express correspondences between different categories and between
images and categories using a unified embedding. Then, we use the latter to
enforce two constraints: symmetric inter-category cycle consistency and a new
asymmetric image-to-category cycle consistency. Without any manual annotations
for the inter-category correspondences, we obtain state-of-the-art alignment
results, outperforming dedicated methods for matching 3D shapes. Moreover, the
new model is also better at the task of dense pose prediction than prior work.
