  Understanding the 3D world without supervision is currently a major challenge
in computer vision as the annotations required to supervise deep networks for
tasks in this domain are expensive to obtain on a large scale. In this paper,
we address the problem of unsupervised viewpoint estimation. We formulate this
as a self-supervised learning task, where image reconstruction provides the
supervision needed to predict the camera viewpoint. Specifically, we make use
of pairs of images of the same object at training time, from unknown
viewpoints, to self-supervise training by combining the viewpoint information
from one image with the appearance information from the other. We demonstrate
that using a perspective spatial transformer allows efficient viewpoint
learning, outperforming existing unsupervised approaches on synthetic data, and
obtains competitive results on the challenging PASCAL3D+ dataset.
