  The ability to edit materials of objects in images is desirable by many
content creators. However, this is an extremely challenging task as it requires
to disentangle intrinsic physical properties of an image. We propose an
end-to-end network architecture that replicates the forward image formation
process to accomplish this task. Specifically, given a single image, the
network first predicts intrinsic properties, i.e. shape, illumination, and
material, which are then provided to a rendering layer. This layer performs
in-network image synthesis, thereby enabling the network to understand the
physics behind the image formation process. The proposed rendering layer is
fully differentiable, supports both diffuse and specular materials, and thus
can be applicable in a variety of problem settings. We demonstrate a rich set
of visually plausible material editing examples and provide an extensive
comparative study.
