  Facial image manipulation has achieved great progress in recent years.
However, previous methods either operate on a predefined set of face attributes
or leave users little freedom to interactively manipulate images. To overcome
these drawbacks, we propose a novel framework termed MaskGAN, enabling diverse
and interactive face manipulation. Our key insight is that semantic masks serve
as a suitable intermediate representation for flexible face manipulation with
fidelity preservation. MaskGAN has two main components: 1) Dense Mapping
Network (DMN) and 2) Editing Behavior Simulated Training (EBST). Specifically,
DMN learns style mapping between a free-form user modified mask and a target
image, enabling diverse generation results. EBST models the user editing
behavior on the source mask, making the overall framework more robust to
various manipulated inputs. Specifically, it introduces dual-editing
consistency as the auxiliary supervision signal. To facilitate extensive
studies, we construct a large-scale high-resolution face dataset with
fine-grained mask annotations named CelebAMask-HQ. MaskGAN is comprehensively
evaluated on two challenging tasks: attribute transfer and style copy,
demonstrating superior performance over other state-of-the-art methods. The
code, models, and dataset are available at
https://github.com/switchablenorms/CelebAMask-HQ.
