  In this paper, we propose an improved quantitative evaluation framework for
Generative Adversarial Networks (GANs) on generating domain-specific images,
where we improve conventional evaluation methods on two levels: the feature
representation and the evaluation metric. Unlike most existing evaluation
frameworks which transfer the representation of ImageNet inception model to map
images onto the feature space, our framework uses a specialized encoder to
acquire fine-grained domain-specific representation. Moreover, for datasets
with multiple classes, we propose Class-Aware Frechet Distance (CAFD), which
employs a Gaussian mixture model on the feature space to better fit the
multi-manifold feature distribution. Experiments and analysis on both the
feature level and the image level were conducted to demonstrate improvements of
our proposed framework over the recently proposed state-of-the-art FID method.
To our best knowledge, we are the first to provide counter examples where FID
gives inconsistent results with human judgments. It is shown in the experiments
that our framework is able to overcome the shortness of FID and improves
robustness. Code will be made available.
