  Recent cutting-edge feature aggregation paradigms for video object detection
rely on inferring feature correspondence. The feature correspondence estimation
problem is fundamentally difficult due to poor image quality, motion blur, etc,
and the results of feature correspondence estimation are unstable. To avoid the
problem, we propose a simple but effective feature aggregation framework which
operates on the object proposal-level. It learns to enhance each proposal's
feature via modeling semantic and spatio-temporal relationships among object
proposals from both within a frame and across adjacent frames. Experiments are
carried out on the ImageNet VID dataset. Without any bells and whistles, our
method obtains 80.3\% mAP on the ImageNet VID dataset, which is superior over
the previous state-of-the-arts. The proposed feature aggregation mechanism
improves the single frame Faster RCNN baseline by 5.8% mAP. Besides, under the
setting of no temporal post-processing, our method outperforms the previous
state-of-the-art by 1.4% mAP.
