  We present a deep learning solution for estimating the incident illumination
at any 3D location within a scene from an input narrow-baseline stereo image
pair. Previous approaches for predicting global illumination from images either
predict just a single illumination for the entire scene, or separately estimate
the illumination at each 3D location without enforcing that the predictions are
consistent with the same 3D scene. Instead, we propose a deep learning model
that estimates a 3D volumetric RGBA model of a scene, including content outside
the observed field of view, and then uses standard volume rendering to estimate
the incident illumination at any 3D location within that volume. Our model is
trained without any ground truth 3D data and only requires a held-out
perspective view near the input stereo pair and a spherical panorama taken
within each scene as supervision, as opposed to prior methods for
spatially-varying lighting estimation, which require ground truth scene
geometry for training. We demonstrate that our method can predict consistent
spatially-varying lighting that is convincing enough to plausibly relight and
insert highly specular virtual objects into real images.
