  Video classification is a challenging task in computer vision. Although Deep
Neural Networks (DNNs) have achieved excellent performance in video
classification, recent research shows adding imperceptible perturbations to
clean videos can make the well-trained models output wrong labels with high
confidence. In this paper, we propose an effective defense framework to
characterize and defend adversarial videos. The proposed method contains two
phases: (1) adversarial video detection using temporal consistency between
adjacent frames, and (2) adversarial perturbation reduction via denoisers in
the spatial and temporal domains respectively. Specifically, because of the
linear nature of DNNs, the imperceptible perturbations will enlarge with the
increasing of DNNs depth, which leads to the inconsistency of DNNs output
between adjacent frames. However, the benign video frames often have the same
outputs with their neighbor frames owing to the slight changes. Based on this
observation, we can distinguish between adversarial videos and benign videos.
After that, we utilize different defense strategies against different attacks.
We propose the temporal defense, which reconstructs the polluted frames with
their temporally neighbor clean frames, to deal with the adversarial videos
with sparse polluted frames. For the videos with dense polluted frames, we use
an efficient adversarial denoiser to process each frame in the spatial domain,
and thus purify the perturbations (we call it as spatial defense). A series of
experiments conducted on the UCF-101 dataset demonstrate that the proposed
method significantly improves the robustness of video classifiers against
adversarial attacks.
