  Despite significant advancements of deep learning-based forgery detectors for
distinguishing manipulated deepfake images, most detection approaches suffer
from moderate to significant performance degradation with low-quality
compressed deepfake images. Because of the limited information in low-quality
images, detecting low-quality deepfake remains an important challenge. In this
work, we apply frequency domain learning and optimal transport theory in
knowledge distillation (KD) to specifically improve the detection of
low-quality compressed deepfake images. We explore transfer learning capability
in KD to enable a student network to learn discriminative features from
low-quality images effectively. In particular, we propose the Attention-based
Deepfake detection Distiller (ADD), which consists of two novel distillations:
1) frequency attention distillation that effectively retrieves the removed
high-frequency components in the student network, and 2) multi-view attention
distillation that creates multiple attention vectors by slicing the teacher's
and student's tensors under different views to transfer the teacher tensor's
distribution to the student more efficiently. Our extensive experimental
results demonstrate that our approach outperforms state-of-the-art baselines in
detecting low-quality compressed deepfake images.
