  In this work, we address the problem of jointly estimating albedo, normals,
depth and 3D spatially-varying lighting from a single image. Most existing
methods formulate the task as image-to-image translation, ignoring the 3D
properties of the scene. However, indoor scenes contain complex 3D light
transport where a 2D representation is insufficient. In this paper, we propose
a unified, learning-based inverse rendering framework that formulates 3D
spatially-varying lighting. Inspired by classic volume rendering techniques, we
propose a novel Volumetric Spherical Gaussian representation for lighting,
which parameterizes the exitant radiance of the 3D scene surfaces on a voxel
grid. We design a physics based differentiable renderer that utilizes our 3D
lighting representation, and formulates the energy-conserving image formation
process that enables joint training of all intrinsic properties with the
re-rendering constraint. Our model ensures physically correct predictions and
avoids the need for ground-truth HDR lighting which is not easily accessible.
Experiments show that our method outperforms prior works both quantitatively
and qualitatively, and is capable of producing photorealistic results for AR
applications such as virtual object insertion even for highly specular objects.
