  This work addresses the problem of anonymizing the identity of faces in a
dataset of images, such that the privacy of those depicted is not violated,
while at the same time the dataset is useful for downstream task such as for
training machine learning models. To the best of our knowledge, we are the
first to explicitly address this issue and deal with two major drawbacks of the
existing state-of-the-art approaches, namely that they (i) require the costly
training of additional, purpose-trained neural networks, and/or (ii) fail to
retain the facial attributes of the original images in the anonymized
counterparts, the preservation of which is of paramount importance for their
use in downstream tasks. We accordingly present a task-agnostic anonymization
procedure that directly optimizes the images' latent representation in the
latent space of a pre-trained GAN. By optimizing the latent codes directly, we
ensure both that the identity is of a desired distance away from the original
(with an identity obfuscation loss), whilst preserving the facial attributes
(using a novel feature-matching loss in FaRL's deep feature space). We
demonstrate through a series of both qualitative and quantitative experiments
that our method is capable of anonymizing the identity of the images whilst --
crucially -- better-preserving the facial attributes. We make the code and the
pre-trained models publicly available at: https://github.com/chi0tzp/FALCO.
