  This paper addresses the temporal sentence grounding (TSG). Although existing
methods have made decent achievements in this task, they not only severely rely
on abundant video-query paired data for training, but also easily fail into the
dataset distribution bias. To alleviate these limitations, we introduce a novel
Equivariant Consistency Regulation Learning (ECRL) framework to learn more
discriminative query-related frame-wise representations for each video, in a
self-supervised manner. Our motivation comes from that the temporal boundary of
the query-guided activity should be consistently predicted under various
video-level transformations. Concretely, we first design a series of
spatio-temporal augmentations on both foreground and background video segments
to generate a set of synthetic video samples. In particular, we devise a
self-refine module to enhance the completeness and smoothness of the augmented
video. Then, we present a novel self-supervised consistency loss (SSCL) applied
on the original and augmented videos to capture their invariant query-related
semantic by minimizing the KL-divergence between the sequence similarity of two
videos and a prior Gaussian distribution of timestamp distance. At last, a
shared grounding head is introduced to predict the transform-equivariant
query-guided segment boundaries for both the original and augmented videos.
Extensive experiments on three challenging datasets (ActivityNet, TACoS, and
Charades-STA) demonstrate both effectiveness and efficiency of our proposed
ECRL framework.
