  In this paper we present, to the best of our knowledge, the first method to
learn a generative model of 3D shapes from natural images in a fully
unsupervised way. For example, we do not use any ground truth 3D or 2D
annotations, stereo video, and ego-motion during the training. Our approach
follows the general strategy of Generative Adversarial Networks, where an image
generator network learns to create image samples that are realistic enough to
fool a discriminator network into believing that they are natural images. In
contrast, in our approach the image generation is split into 2 stages. In the
first stage a generator network outputs 3D objects. In the second, a
differentiable renderer produces an image of the 3D objects from random
viewpoints. The key observation is that a realistic 3D object should yield a
realistic rendering from any plausible viewpoint. Thus, by randomizing the
choice of the viewpoint our proposed training forces the generator network to
learn an interpretable 3D representation disentangled from the viewpoint. In
this work, a 3D representation consists of a triangle mesh and a texture map
that is used to color the triangle surface by using the UV-mapping technique.
We provide analysis of our learning approach, expose its ambiguities and show
how to overcome them. Experimentally, we demonstrate that our method can learn
realistic 3D shapes of faces by using only the natural images of the FFHQ
dataset.
