  The existing action tubelet detectors often depend on heuristic anchor design
and placement, which might be computationally expensive and sub-optimal for
precise localization. In this paper, we present a conceptually simple,
computationally efficient, and more precise action tubelet detection framework,
termed as MovingCenter Detector (MOC-detector), by treating an action instance
as a trajectory of moving points. Based on the insight that movement
information could simplify and assist action tubelet detection, our
MOC-detector is composed of three crucial head branches: (1) Center Branch for
instance center detection and action recognition, (2) Movement Branch for
movement estimation at adjacent frames to form trajectories of moving points,
(3) Box Branch for spatial extent detection by directly regressing bounding box
size at each estimated center. These three branches work together to generate
the tubelet detection results, which could be further linked to yield
video-level tubes with a matching strategy. Our MOC-detector outperforms the
existing state-of-the-art methods for both metrics of frame-mAP and video-mAP
on the JHMDB and UCF101-24 datasets. The performance gap is more evident for
higher video IoU, demonstrating that our MOC-detector is particularly effective
for more precise action detection. We provide the code at
https://github.com/MCG-NJU/MOC-Detector.
