  This paper addresses the problem of how to exploit spatio-temporal
information available in videos to improve the object detection precision. We
propose a two stage object detector called FANet based on short-term
spatio-temporal feature aggregation to give a first detection set, and
long-term object linking to refine these detections. Firstly, we generate a set
of short tubelet proposals containing the object in $N$ consecutive frames.
Then, we aggregate RoI pooled deep features through the tubelet using a
temporal pooling operator that summarizes the information with a fixed size
output independent of the number of input frames. On top of that, we define a
double head implementation that we feed with spatio-temporal aggregated
information for spatio-temporal object classification, and with spatial
information extracted from the current frame for object localization and
spatial classification. Furthermore, we also specialize each head branch
architecture to better perform in each task taking into account the input data.
Finally, a long-term linking method builds long tubes using the previously
calculated short tubelets to overcome detection errors. We have evaluated our
model in the widely used ImageNet VID dataset achieving a 80.9% mAP, which is
the new state-of-the-art result for single models. Also, in the challenging
small object detection dataset USC-GRAD-STDdb, our proposal outperforms the
single frame baseline by 5.4% mAP.
