  Despite the fact that DeepFake forgery detection algorithms have achieved
impressive performance on known manipulations, they often face disastrous
performance degradation when generalized to an unseen manipulation. Some recent
works show improvement in generalization but rely on features fragile to image
distortions such as compression. To this end, we propose Diff-ID, a concise and
effective approach that explains and measures the identity loss induced by
facial manipulations. When testing on an image of a specific person, Diff-ID
utilizes an authentic image of that person as a reference and aligns them to
the same identity-insensitive attribute feature space by applying a
face-swapping generator. We then visualize the identity loss between the test
and the reference image from the image differences of the aligned pairs, and
design a custom metric to quantify the identity loss. The metric is then proved
to be effective in distinguishing the forgery images from the real ones.
Extensive experiments show that our approach achieves high detection
performance on DeepFake images and state-of-the-art generalization ability to
unknown forgery methods, while also being robust to image distortions.
