  Deep convolutional neural networks have shown remarkable results on multiple
detection tasks. Despite the significant progress, the performance of such
detectors are often assessed in public benchmarks under non-realistic
conditions. Specifically, impact of conventional distortions and processing
operations such as compression, noise, and enhancement are not sufficiently
studied. This paper proposes a rigorous framework to assess performance of
learning-based detectors in more realistic situations. An illustrative example
is shown under deepfake detection context. Inspired by the assessment results,
a data augmentation strategy based on natural image degradation process is
designed, which significantly improves the generalization ability of two
deepfake detectors.
