  The recent renaissance in generative models, driven primarily by the advent
of diffusion models and iterative improvement in GAN methods, has enabled many
creative applications. However, each advancement is also accompanied by a rise
in the potential for misuse. In the arena of the deepfake generation, this is a
key societal issue. In particular, the ability to modify segments of videos
using such generative techniques creates a new paradigm of deepfakes which are
mostly real videos altered slightly to distort the truth. This paradigm has
been under-explored by the current deepfake detection methods in the academic
literature. In this paper, we present a deepfake detection method that can
address this issue by performing deepfake prediction at the frame and video
levels. To facilitate testing our method, we prepared a new benchmark dataset
where videos have both real and fake frame sequences with very subtle
transitions. We provide a benchmark on the proposed dataset with our detection
method which utilizes the Vision Transformer based on Scaling and Shifting to
learn spatial features, and a Timeseries Transformer to learn temporal features
of the videos to help facilitate the interpretation of possible deepfakes.
Extensive experiments on a variety of deepfake generation methods show
excellent results by the proposed method on temporal segmentation and classical
video-level predictions as well. In particular, the paradigm we address will
form a powerful tool for the moderation of deepfakes, where human oversight can
be better targeted to the parts of videos suspected of being deepfakes. All
experiments can be reproduced at:
github.com/rgb91/temporal-deepfake-segmentation.
