  We are interested in learning visual representations which allow for 3D
manipulations of visual objects based on a single 2D image. We cast this into
an image-to-image transformation task, and propose Iterative Generative
Adversarial Networks (IterGANs) which iteratively transform an input image into
an output image. Our models learn a visual representation that can be used for
objects seen in training, but also for never seen objects. Since object
manipulation requires a full understanding of the geometry and appearance of
the object, our IterGANs learn an implicit 3D model and a full appearance model
of the object, which are both inferred from a single (test) image. Two
advantages of IterGANs are that the intermediate generated images can be used
for an additional supervision signal, even in an unsupervised fashion, and that
the number of iterations can be used as a control signal to steer the
transformation. Experiments on rotated objects and scenes show how IterGANs
help with the generation process.
