  Recently, multiple formulations of vision problems as probabilistic
inversions of generative models based on computer graphics have been proposed.
However, applications to 3D perception from natural images have focused on
low-dimensional latent scenes, due to challenges in both modeling and
inference. Accounting for the enormous variability in 3D object shape and 2D
appearance via realistic generative models seems intractable, as does inverting
even simple versions of the many-to-many computations that link 3D scenes to 2D
images. This paper proposes and evaluates an approach that addresses key
aspects of both these challenges. We show that it is possible to solve
challenging, real-world 3D vision problems by approximate inference in
generative models for images based on rendering the outputs of probabilistic
CAD (PCAD) programs. Our PCAD object geometry priors generate deformable 3D
meshes corresponding to plausible objects and apply affine transformations to
place them in a scene. Image likelihoods are based on similarity in a feature
space based on standard mid-level image representations from the vision
literature. Our inference algorithm integrates single-site and locally blocked
Metropolis-Hastings proposals, Hamiltonian Monte Carlo and discriminative
data-driven proposals learned from training data generated from our models. We
apply this approach to 3D human pose estimation and object shape reconstruction
from single images, achieving quantitative and qualitative performance
improvements over state-of-the-art baselines.
