  We present a method for estimating detailed scene illumination using human
faces in a single image. In contrast to previous works that estimate lighting
in terms of low-order basis functions or distant point lights, our technique
estimates illumination at a higher precision in the form of a non-parametric
environment map. Based on the observation that faces can exhibit strong
highlight reflections from a broad range of lighting directions, we propose a
deep neural network for extracting highlights from faces, and then trace these
reflections back to the scene to acquire the environment map. Since real
training data for highlight extraction is very limited, we introduce an
unsupervised scheme for finetuning the network on real images, based on the
consistent diffuse chromaticity of a given face seen in multiple real images.
In tracing the estimated highlights to the environment, we reduce the blurring
effect of skin reflectance on reflected light through a deconvolution
determined by prior knowledge on face material properties. Comparisons to
previous techniques for highlight extraction and illumination estimation show
the state-of-the-art performance of this approach on a variety of indoor and
outdoor scenes.
