  Rendering bridges the gap between 2D vision and 3D scenes by simulating the
physical process of image formation. By inverting such renderer, one can think
of a learning approach to infer 3D information from 2D images. However,
standard graphics renderers involve a fundamental discretization step called
rasterization, which prevents the rendering process to be differentiable, hence
able to be learned. Unlike the state-of-the-art differentiable renderers, which
only approximate the rendering gradient in the back propagation, we propose a
truly differentiable rendering framework that is able to (1) directly render
colorized mesh using differentiable functions and (2) back-propagate efficient
supervision signals to mesh vertices and their attributes from various forms of
image representations, including silhouette, shading and color images. The key
to our framework is a novel formulation that views rendering as an aggregation
function that fuses the probabilistic contributions of all mesh triangles with
respect to the rendered pixels. Such formulation enables our framework to flow
gradients to the occluded and far-range vertices, which cannot be achieved by
the previous state-of-the-arts. We show that by using the proposed renderer,
one can achieve significant improvement in 3D unsupervised single-view
reconstruction both qualitatively and quantitatively. Experiments also
demonstrate that our approach is able to handle the challenging tasks in
image-based shape fitting, which remain nontrivial to existing differentiable
renderers.
