  Facial forgery by deepfakes has raised severe societal concerns. Several
solutions have been proposed by the vision community to effectively combat the
misinformation on the internet via automated deepfake detection systems. Recent
studies have demonstrated that facial analysis-based deep learning models can
discriminate based on protected attributes. For the commercial adoption and
massive roll-out of the deepfake detection technology, it is vital to evaluate
and understand the fairness (the absence of any prejudice or favoritism) of
deepfake detectors across demographic variations such as gender and race. As
the performance differential of deepfake detectors between demographic
subgroups would impact millions of people of the deprived sub-group. This paper
aims to evaluate the fairness of the deepfake detectors across males and
females. However, existing deepfake datasets are not annotated with demographic
labels to facilitate fairness analysis. To this aim, we manually annotated
existing popular deepfake datasets with gender labels and evaluated the
performance differential of current deepfake detectors across gender. Our
analysis on the gender-labeled version of the datasets suggests (a) current
deepfake datasets have skewed distribution across gender, and (b) commonly
adopted deepfake detectors obtain unequal performance across gender with mostly
males outperforming females. Finally, we contributed a gender-balanced and
annotated deepfake dataset, GBDF, to mitigate the performance differential and
to promote research and development towards fairness-aware deep fake detectors.
The GBDF dataset is publicly available at: https://github.com/aakash4305/GBDF
