  As GAN-based video and image manipulation technologies become more
sophisticated and easily accessible, there is an urgent need for effective
deepfake detection technologies. Moreover, various deepfake generation
techniques have emerged over the past few years. While many deepfake detection
methods have been proposed, their performance suffers from new types of
deepfake methods on which they are not sufficiently trained. To detect new
types of deepfakes, the model should learn from additional data without losing
its prior knowledge about deepfakes (catastrophic forgetting), especially when
new deepfakes are significantly different. In this work, we employ the
Representation Learning (ReL) and Knowledge Distillation (KD) paradigms to
introduce a transfer learning-based Feature Representation Transfer Adaptation
Learning (FReTAL) method. We use FReTAL to perform domain adaptation tasks on
new deepfake datasets while minimizing catastrophic forgetting. Our student
model can quickly adapt to new types of deepfake by distilling knowledge from a
pre-trained teacher model and applying transfer learning without using source
domain data during domain adaptation. Through experiments on FaceForensics++
datasets, we demonstrate that FReTAL outperforms all baselines on the domain
adaptation task with up to 86.97% accuracy on low-quality deepfakes.
