  Existing temporal action detection (TAD) methods rely on generating an
overwhelmingly large number of proposals per video. This leads to complex model
designs due to proposal generation and/or per-proposal action instance
evaluation and the resultant high computational cost. In this work, for the
first time, we propose a proposal-free Temporal Action detection model with
Global Segmentation mask (TAGS). Our core idea is to learn a global
segmentation mask of each action instance jointly at the full video length. The
TAGS model differs significantly from the conventional proposal-based methods
by focusing on global temporal representation learning to directly detect local
start and end points of action instances without proposals. Further, by
modeling TAD holistically rather than locally at the individual proposal level,
TAGS needs a much simpler model architecture with lower computational cost.
Extensive experiments show that despite its simpler design, TAGS outperforms
existing TAD methods, achieving new state-of-the-art performance on two
benchmarks. Importantly, it is ~ 20x faster to train and ~1.6x more efficient
for inference. Our PyTorch implementation of TAGS is available at
https://github.com/sauradip/TAGS .
