  This paper proposes learning disentangled but complementary face features
with minimal supervision by face identification. Specifically, we construct an
identity Distilling and Dispelling Autoencoder (D2AE) framework that
adversarially learns the identity-distilled features for identity verification
and the identity-dispelled features to fool the verification system. Thanks to
the design of two-stream cues, the learned disentangled features represent not
only the identity or attribute but the complete input image. Comprehensive
evaluations further demonstrate that the proposed features not only maintain
state-of-the-art identity verification performance on LFW, but also acquire
competitive discriminative power for face attribute recognition on CelebA and
LFWA. Moreover, the proposed system is ready to semantically control the face
generation/editing based on various identities and attributes in an
unsupervised manner.
