  We address the problem of video grounding from natural language queries. The
key challenge in this task is that one training video might only contain a few
annotated starting/ending frames that can be used as positive examples for
model training. Most conventional approaches directly train a binary classifier
using such imbalance data, thus achieving inferior results. The key idea of
this paper is to use the distances between the frame within the ground truth
and the starting (ending) frame as dense supervisions to improve the video
grounding accuracy. Specifically, we design a novel dense regression network
(DRN) to regress the distances from each frame to the starting (ending) frame
of the video segment described by the query. We also propose a simple but
effective IoU regression head module to explicitly consider the localization
quality of the grounding results (i.e., the IoU between the predicted location
and the ground truth). Experimental results show that our approach
significantly outperforms state-of-the-arts on three datasets (i.e.,
Charades-STA, ActivityNet-Captions, and TACoS).
