  Facial forgery detection is a crucial but extremely challenging topic, with
the fast development of forgery techniques making the synthetic artefact highly
indistinguishable. Prior works show that by mining both spatial and frequency
information the forgery detection performance of deep learning models can be
vastly improved. However, leveraging multiple types of information usually
requires more than one branch in the neural network, which makes the model
heavy and cumbersome. Knowledge distillation, as an important technique for
efficient modelling, could be a possible remedy. We find that existing
knowledge distillation methods have difficulties distilling a dual-branch model
into a single-branch model. More specifically, knowledge distillation on both
the spatial and frequency branches has degraded performance than distillation
only on the spatial branch. To handle such problem, we propose a novel
two-in-one knowledge distillation framework which can smoothly merge the
information from a large dual-branch network into a small single-branch
network, with the help of different dedicated feature projectors and the
gradient homogenization technique. Experimental analysis on two datasets,
FaceForensics++ and Celeb-DF, shows that our proposed framework achieves
superior performance for facial forgery detection with much fewer parameters.
