  Face recognition based on the deep convolutional neural networks (CNN) shows
superior accuracy performance attributed to the high discriminative features
extracted. Yet, the security and privacy of the extracted features from deep
learning models (deep features) have been often overlooked. This paper proposes
the reconstruction of face images from deep features without accessing the CNN
network configurations as a constrained optimization problem. Such optimization
minimizes the distance between the features extracted from the original face
image and the reconstructed face image. Instead of directly solving the
optimization problem in the image space, we innovatively reformulate the
problem by looking for a latent vector of a GAN generator, then use it to
generate the face image. The GAN generator serves as a dual role in this novel
framework, i.e., face distribution constraint of the optimization goal and a
face generator. On top of the novel optimization task, we also propose an
attack pipeline to impersonate the target user based on the generated face
image. Our results show that the generated face images can achieve a
state-of-the-art successful attack rate of 98.0\% on LFW under type-I attack @
FAR of 0.1\%. Our work sheds light on the biometric deployment to meet the
privacy-preserving and security policies.
