  Given a portrait image of a person and an environment map of the target
lighting, portrait relighting aims to re-illuminate the person in the image as
if the person appeared in an environment with the target lighting. To achieve
high-quality results, recent methods rely on deep learning. An effective
approach is to supervise the training of deep neural networks with a
high-fidelity dataset of desired input-output pairs, captured with a light
stage. However, acquiring such data requires an expensive special capture rig
and time-consuming efforts, limiting access to only a few resourceful
laboratories. To address the limitation, we propose a new approach that can
perform on par with the state-of-the-art (SOTA) relighting methods without
requiring a light stage. Our approach is based on the realization that a
successful relighting of a portrait image depends on two conditions. First, the
method needs to mimic the behaviors of physically-based relighting. Second, the
output has to be photorealistic. To meet the first condition, we propose to
train the relighting network with training data generated by a virtual light
stage that performs physically-based rendering on various 3D synthetic humans
under different environment maps. To meet the second condition, we develop a
novel synthetic-to-real approach to bring photorealism to the relighting
network output. In addition to achieving SOTA results, our approach offers
several advantages over the prior methods, including controllable glares on
glasses and more temporally-consistent results for relighting videos.
