  This paper addresses the problem of natural language video localization
(NLVL). Almost all existing works follow the "only look once" framework that
exploits a single model to directly capture the complex cross- and self-modal
relations among video-query pairs and retrieve the relevant segment. However,
we argue that these methods have overlooked two indispensable characteristics
of an ideal localization method: 1) Frame-differentiable: considering the
imbalance of positive/negative video frames, it is effective to highlight
positive frames and weaken negative ones during the localization. 2)
Boundary-precise: to predict the exact segment boundary, the model should
capture more fine-grained differences between consecutive frames since their
variations are often smooth. To this end, inspired by how humans perceive and
localize a segment, we propose a two-step human-like framework called
Skimming-Locating-Perusing (SLP). SLP consists of a Skimming-and-Locating (SL)
module and a Bi-directional Perusing (BP) module. The SL module first refers to
the query semantic and selects the best matched frame from the video while
filtering out irrelevant frames. Then, the BP module constructs an initial
segment based on this frame, and dynamically updates it by exploring its
adjacent frames until no frame shares the same activity semantic. Experimental
results on three challenging benchmarks show that our SLP is superior to the
state-of-the-art methods and localizes more precise segment boundaries.
