  We propose a new approach for editing face images, which enables numerous
exciting applications including face relighting, makeup transfer and face
detail editing. Our face edits are based on a visual representation, which
includes geometry, face segmentation, albedo, illumination and detail map. To
recover our visual representation, we start by estimating geometry using a
morphable face model, then decompose the face image to recover the albedo, and
then shade the geometry with the albedo and illumination. The residual between
our shaded geometry and the input image produces our detail map, which carries
high frequency information that is either insufficiently or incorrectly
captured by our shading process. By manipulating the detail map, we can edit
face images with reality and identity preserved. Our representation allows
various applications. First, it allows a user to directly manipulate various
illumination. Second, it allows non-parametric makeup transfer with input
face's distinctive identity features preserved. Third, it allows non-parametric
modifications to the face appearance by transferring details. For face
relighting and detail editing, we evaluate via a user study and our method
outperforms other methods. For makeup transfer, we evaluate via an online
attractiveness evaluation system, and can reliably make people look younger and
more attractive. We also show extensive qualitative comparisons to existing
methods, and have significant improvements over previous techniques.
