  Transferring image-based object detectors to the domain of videos remains a
challenging problem. Previous efforts mostly exploit optical flow to propagate
features across frames, aiming to achieve a good trade-off between accuracy and
efficiency. However, introducing an extra model to estimate optical flow can
significantly increase the overall model size. The gap between optical flow and
high-level features can also hinder it from establishing spatial correspondence
accurately. Instead of relying on optical flow, this paper proposes a novel
module called Progressive Sparse Local Attention (PSLA), which establishes the
spatial correspondence between features across frames in a local region with
progressively sparser stride and uses the correspondence to propagate features.
Based on PSLA, Recursive Feature Updating (RFU) and Dense Feature Transforming
(DenseFT) are proposed to model temporal appearance and enrich feature
representation respectively in a novel video object detection framework.
Experiments on ImageNet VID show that our method achieves the best accuracy
compared to existing methods with smaller model size and acceptable runtime
speed.
