  In low-level video analyses, effective representations are important to
derive the correspondences between video frames. These representations have
been learned in a self-supervised fashion from unlabeled images or videos,
using carefully designed pretext tasks in some recent studies. However, the
previous work concentrates on either spatial-discriminative features or
temporal-repetitive features, with little attention to the synergy between
spatial and temporal cues. To address this issue, we propose a
spatial-then-temporal self-supervised learning method. Specifically, we firstly
extract spatial features from unlabeled images via contrastive learning, and
secondly enhance the features by exploiting the temporal cues in unlabeled
videos via reconstructive learning. In the second step, we design a global
correlation distillation loss to ensure the learning not to forget the spatial
cues, and a local correlation distillation loss to combat the temporal
discontinuity that harms the reconstruction. The proposed method outperforms
the state-of-the-art self-supervised methods, as established by the
experimental results on a series of correspondence-based video analysis tasks.
Also, we performed ablation studies to verify the effectiveness of the two-step
design as well as the distillation losses.
