  Given a set of images of a scene, the re-rendering of this scene from novel
views and lighting conditions is an important and challenging problem in
Computer Vision and Graphics. On the one hand, most existing works in Computer
Vision usually impose many assumptions regarding the image formation process,
e.g. direct illumination and predefined materials, to make scene parameter
estimation tractable. On the other hand, mature Computer Graphics tools allow
modeling of complex photo-realistic light transport given all the scene
parameters. Combining these approaches, we propose a method for scene
relighting under novel views by learning a neural precomputed radiance transfer
function, which implicitly handles global illumination effects using novel
environment maps. Our method can be solely supervised on a set of real images
of the scene under a single unknown lighting condition. To disambiguate the
task during training, we tightly integrate a differentiable path tracer in the
training process and propose a combination of a synthesized OLAT and a real
image loss. Results show that the recovered disentanglement of scene parameters
improves significantly over the current state of the art and, thus, also our
re-rendering results are more realistic and accurate.
