  Despite their excellent performance, state-of-the-art computer vision models
often fail when they encounter adversarial examples. Video perception models
tend to be more fragile under attacks, because the adversary has more places to
manipulate in high-dimensional data. In this paper, we find one reason for
video models' vulnerability is that they fail to perceive the correct motion
under adversarial perturbations. Inspired by the extensive evidence that motion
is a key factor for the human visual system, we propose to correct what the
model sees by restoring the perceived motion information. Since motion
information is an intrinsic structure of the video data, recovering motion
signals can be done at inference time without any human annotation, which
allows the model to adapt to unforeseen, worst-case inputs. Visualizations and
empirical experiments on UCF-101 and HMDB-51 datasets show that restoring
motion information in deep vision models improves adversarial robustness. Even
under adaptive attacks where the adversary knows our defense, our algorithm is
still effective. Our work provides new insight into robust video perception
algorithms by using intrinsic structures from the data. Our webpage is
available at https://motion4robust.cs.columbia.edu.
