  AI-generated faces have enriched human life, such as entertainment,
education, and art. However, they also pose misuse risks. Therefore, detecting
AI-generated faces becomes crucial, yet current detectors show biased
performance across different demographic groups. Mitigating biases can be done
by designing algorithmic fairness methods, which usually require
demographically annotated face datasets for model training. However, no
existing dataset comprehensively encompasses both demographic attributes and
diverse generative methods, which hinders the development of fair detectors for
AI-generated faces. In this work, we introduce the AI-Face dataset, the first
million-scale demographically annotated AI-generated face image dataset,
including real faces, faces from deepfake videos, and faces generated by
Generative Adversarial Networks and Diffusion Models. Based on this dataset, we
conduct the first comprehensive fairness benchmark to assess various AI face
detectors and provide valuable insights and findings to promote the future fair
design of AI face detectors. Our AI-Face dataset and benchmark code are
publicly available at https://github.com/Purdue-M2/AI-Face-FairnessBench.
