  We present a learning-based approach to relight a single image of Lambertian
and low-frequency specular objects. Our method enables inserting objects from
photographs into new scenes and relighting them under the new environment
lighting, which is essential for AR applications. To relight the object, we
solve both inverse rendering and re-rendering. To resolve the ill-posed inverse
rendering, we propose a weakly-supervised method by a low-rank constraint. To
facilitate the weakly-supervised training, we contribute Relit, a large-scale
(750K images) dataset of videos with aligned objects under changing
illuminations. For re-rendering, we propose a differentiable specular rendering
layer to render low-frequency non-Lambertian materials under various
illuminations of spherical harmonics. The whole pipeline is end-to-end and
efficient, allowing for a mobile app implementation of AR object insertion.
Extensive evaluations demonstrate that our method achieves state-of-the-art
performance. Project page: https://renjiaoyi.github.io/relighting/.
