  We explore the task of Canonical Surface Mapping (CSM). Specifically, given
an image, we learn to map pixels on the object to their corresponding locations
on an abstract 3D model of the category. But how do we learn such a mapping? A
supervised approach would require extensive manual labeling which is not
scalable beyond a few hand-picked categories. Our key insight is that the CSM
task (pixel to 3D), when combined with 3D projection (3D to pixel), completes a
cycle. Hence, we can exploit a geometric cycle consistency loss, thereby
allowing us to forgo the dense manual supervision. Our approach allows us to
train a CSM model for a diverse set of classes, without sparse or dense
keypoint annotation, by leveraging only foreground mask labels for training. We
show that our predictions also allow us to infer dense correspondence between
two images, and compare the performance of our approach against several methods
that predict correspondence by leveraging varying amount of supervision.
