  Temporally localizing actions in a video is a fundamental challenge in video
understanding. Most existing approaches have often drawn inspiration from image
object detection and extended the advances, e.g., SSD and Faster R-CNN, to
produce temporal locations of an action in a 1D sequence. Nevertheless, the
results can suffer from robustness problem due to the design of predetermined
temporal scales, which overlooks the temporal structure of an action and limits
the utility on detecting actions with complex variations. In this paper, we
propose to address the problem by introducing Gaussian kernels to dynamically
optimize temporal scale of each action proposal. Specifically, we present
Gaussian Temporal Awareness Networks (GTAN) --- a new architecture that novelly
integrates the exploitation of temporal structure into an one-stage action
localization framework. Technically, GTAN models the temporal structure through
learning a set of Gaussian kernels, each for a cell in the feature maps. Each
Gaussian kernel corresponds to a particular interval of an action proposal and
a mixture of Gaussian kernels could further characterize action proposals with
various length. Moreover, the values in each Gaussian curve reflect the
contextual contributions to the localization of an action proposal. Extensive
experiments are conducted on both THUMOS14 and ActivityNet v1.3 datasets, and
superior results are reported when comparing to state-of-the-art approaches.
More remarkably, GTAN achieves 1.9% and 1.1% improvements in mAP on testing set
of the two datasets.
