  Image relighting is the task of showing what a scene from a source image
would look like if illuminated differently. Inverse graphics schemes recover an
explicit representation of geometry and a set of chosen intrinsics, then
relight with some form of renderer. However error control for inverse graphics
is difficult, and inverse graphics methods can represent only the effects of
the chosen intrinsics. This paper describes a relighting method that is
entirely data-driven, where intrinsics and lighting are each represented as
latent variables. Our approach produces SOTA relightings of real scenes, as
measured by standard metrics. We show that albedo can be recovered from our
latent intrinsics without using any example albedos, and that the albedos
recovered are competitive with SOTA methods.
