  Reconstructing the underlying 3D surface of an object from a single image is
a challenging problem that has received extensive attention from the computer
vision community. Many learning-based approaches tackle this problem by
learning a 3D shape prior from either ground truth 3D data or multi-view
observations. To achieve state-of-the-art results, these methods assume that
the objects are specified with respect to a fixed canonical coordinate frame,
where instances of the same category are perfectly aligned. In this work, we
present a new method for joint category-specific 3D reconstruction and object
pose estimation from a single image. We show that one can leverage shape priors
learned on purely synthetic 3D data together with a point cloud pose
canonicalization method to achieve high-quality 3D reconstruction in the wild.
Given a single depth image at test time, we first transform this partial point
cloud into a learned canonical frame. Then, we use a neural deformation field
to reconstruct the 3D surface of the object. Finally, we jointly optimize
object pose and 3D shape to fit the partial depth observation. Our approach
achieves state-of-the-art reconstruction performance across several real-world
datasets, even when trained only on synthetic data. We further show that our
method generalizes to different input modalities, from dense depth images to
sparse and noisy LIDAR scans.
