  Single-view 3D shape reconstruction is an important but challenging problem,
mainly for two reasons. First, as shape annotation is very expensive to
acquire, current methods rely on synthetic data, in which ground-truth 3D
annotation is easy to obtain. However, this results in domain adaptation
problem when applied to natural images. The second challenge is that there are
multiple shapes that can explain a given 2D image. In this paper, we propose a
framework to improve over these challenges using adversarial training. On one
hand, we impose domain confusion between natural and synthetic image
representations to reduce the distribution gap. On the other hand, we impose
the reconstruction to be `realistic' by forcing it to lie on a (learned)
manifold of realistic object shapes. Our experiments show that these
constraints improve performance by a large margin over baseline reconstruction
models. We achieve results competitive with the state of the art with a much
simpler architecture.
