  Deepfake videos present an increasing threat to society with potentially
negative impact on criminal justice, democracy, and personal safety and
privacy. Meanwhile, detecting deepfakes, at scale, remains a very challenging
task that often requires labeled training data from existing deepfake
generation methods. Further, even the most accurate supervised deepfake
detection methods do not generalize to deepfakes generated using new generation
methods. In this paper, we propose a novel unsupervised method for detecting
deepfake videos by directly identifying intra-modal and cross-modal
inconsistency between video segments. The fundamental hypothesis behind the
proposed detection method is that motion or identity inconsistencies are
inevitable in deepfake videos. We will mathematically and empirically support
this hypothesis, and then proceed to constructing our method grounded in our
theoretical analysis. Our proposed method outperforms prior state-of-the-art
unsupervised deepfake detection methods on the challenging FakeAVCeleb dataset,
and also has several additional advantages: it is scalable because it does not
require pristine (real) samples for each identity during inference and
therefore can apply to arbitrarily many identities, generalizable because it is
trained only on real videos and therefore does not rely on a particular
deepfake method, reliable because it does not rely on any likelihood estimation
in high dimensions, and explainable because it can pinpoint the exact location
of modality inconsistencies which are then verifiable by a human expert.
