  We present a Temporal Context Network (TCN) for precise temporal localization
of human activities. Similar to the Faster-RCNN architecture, proposals are
placed at equal intervals in a video which span multiple temporal scales. We
propose a novel representation for ranking these proposals. Since pooling
features only inside a segment is not sufficient to predict activity
boundaries, we construct a representation which explicitly captures context
around a proposal for ranking it. For each temporal segment inside a proposal,
features are uniformly sampled at a pair of scales and are input to a temporal
convolutional neural network for classification. After ranking proposals,
non-maximum suppression is applied and classification is performed to obtain
final detections. TCN outperforms state-of-the-art methods on the ActivityNet
dataset and the THUMOS14 dataset.
