  Face forgery detection is essential in combating malicious digital face
attacks. Previous methods mainly rely on prior expert knowledge to capture
specific forgery clues, such as noise patterns, blending boundaries, and
frequency artifacts. However, these methods tend to get trapped in local
optima, resulting in limited robustness and generalization capability. To
address these issues, we propose a novel Critical Forgery Mining (CFM)
framework, which can be flexibly assembled with various backbones to boost
their generalization and robustness performance. Specifically, we first build a
fine-grained triplet and suppress specific forgery traces through prior
knowledge-agnostic data augmentation. Subsequently, we propose a fine-grained
relation learning prototype to mine critical information in forgeries through
instance and local similarity-aware losses. Moreover, we design a novel
progressive learning controller to guide the model to focus on principal
feature components, enabling it to learn critical forgery features in a
coarse-to-fine manner. The proposed method achieves state-of-the-art forgery
detection performance under various challenging evaluation settings.
