  The light stage has been widely used in computer graphics for the past two
decades, primarily to enable the relighting of human faces. By capturing the
appearance of the human subject under different light sources, one obtains the
light transport matrix of that subject, which enables image-based relighting in
novel environments. However, due to the finite number of lights in the stage,
the light transport matrix only represents a sparse sampling on the entire
sphere. As a consequence, relighting the subject with a point light or a
directional source that does not coincide exactly with one of the lights in the
stage requires interpolation and resampling the images corresponding to nearby
lights, and this leads to ghosting shadows, aliased specularities, and other
artifacts. To ameliorate these artifacts and produce better results under
arbitrary high-frequency lighting, this paper proposes a learning-based
solution for the "super-resolution" of scans of human faces taken from a light
stage. Given an arbitrary "query" light direction, our method aggregates the
captured images corresponding to neighboring lights in the stage, and uses a
neural network to synthesize a rendering of the face that appears to be
illuminated by a "virtual" light source at the query location. This neural
network must circumvent the inherent aliasing and regularity of the light stage
data that was used for training, which we accomplish through the use of
regularized traditional interpolation methods within our network. Our learned
model is able to produce renderings for arbitrary light directions that exhibit
realistic shadows and specular highlights, and is able to generalize across a
wide variety of subjects.
