  Recent self-supervised video representation learning methods focus on
maximizing the similarity between multiple augmented views from the same video
and largely rely on the quality of generated views. However, most existing
methods lack a mechanism to prevent representation learning from bias towards
static information in the video. In this paper, we propose frequency
augmentation (FreqAug), a spatio-temporal data augmentation method in the
frequency domain for video representation learning. FreqAug stochastically
removes specific frequency components from the video so that learned
representation captures essential features more from the remaining information
for various downstream tasks. Specifically, FreqAug pushes the model to focus
more on dynamic features rather than static features in the video via dropping
spatial or temporal low-frequency components. To verify the generality of the
proposed method, we experiment with FreqAug on multiple self-supervised
learning frameworks along with standard augmentations. Transferring the
improved representation to five video action recognition and two temporal
action localization downstream tasks shows consistent improvements over
baselines.
