  With the rise of deep learning, generative models have enabled the creation
of highly realistic synthetic images, presenting challenges due to their
potential misuse. While research in Deepfake detection has grown rapidly in
response, many detection methods struggle with unseen Deepfakes generated by
new synthesis techniques. To address this generalisation challenge, we propose
a novel Deepfake detection approach by adapting the Foundation Models with rich
information encoded inside, specifically using the image encoder from CLIP
which has demonstrated strong zero-shot capability for downstream tasks.
Inspired by the recent advances of parameter efficient fine-tuning, we propose
a novel side-network-based decoder to extract spatial and temporal cues from
the given video clip, with the promotion of the Facial Component Guidance (FCG)
to encourage the spatial feature to include features of key facial parts for
more robust and general Deepfake detection. Through extensive cross-dataset
evaluations, our approach exhibits superior effectiveness in identifying unseen
Deepfake samples, achieving notable performance improvement even with limited
training samples and manipulation types. Our model secures an average
performance enhancement of 0.9\% AUROC in cross-dataset assessments comparing
with state-of-the-art methods, especially a significant lead of achieving 4.4\%
improvement on the challenging DFDC dataset.
