  The task of temporal grounding aims to locate video moment in an untrimmed
video, with a given sentence query. This paper for the first time investigates
some superficial biases that are specific to the temporal grounding task, and
proposes a novel targeted solution. Most alarmingly, we observe that existing
temporal ground models heavily rely on some biases (e.g., high preference on
frequent concepts or certain temporal intervals) in the visual modal. This
leads to inferior performance when generalizing the model in cross-scenario
test setting. To this end, we propose a novel method called Debiased Temporal
Language Localizer (DebiasTLL) to prevent the model from naively memorizing the
biases and enforce it to ground the query sentence based on true inter-modal
relationship. Debias-TLL simultaneously trains two models. By our design, a
large discrepancy of these two models' predictions when judging a sample
reveals higher probability of being a biased sample. Harnessing the informative
discrepancy, we devise a data re-weighing scheme for mitigating the data
biases. We evaluate the proposed model in cross-scenario temporal grounding,
where the train / test data are heterogeneously sourced. Experiments show
large-margin superiority of the proposed method in comparison with
state-of-the-art competitors.
