  Due to the successful development of deep image generation technology,
forgery detection plays a more important role in social and economic security.
Racial bias has not been explored thoroughly in the deep forgery detection
field. In the paper, we first contribute a dedicated dataset called the Fair
Forgery Detection (FairFD) dataset, where we prove the racial bias of public
state-of-the-art (SOTA) methods. Different from existing forgery detection
datasets, the self-construct FairFD dataset contains a balanced racial ratio
and diverse forgery generation images with the largest-scale subjects.
Additionally, we identify the problems with naive fairness metrics when
benchmarking forgery detection models. To comprehensively evaluate fairness, we
design novel metrics including Approach Averaged Metric and Utility Regularized
Metric, which can avoid deceptive results. Extensive experiments conducted with
nine representative forgery detection models demonstrate the value of the
proposed dataset and the reasonability of the designed fairness metrics. We
also conduct more in-depth analyses to offer more insights to inspire
researchers in the community.
