  Object detection in videos has drawn increasing attention since it is more
practical in real scenarios. Most of the deep learning methods use CNNs to
process each decoded frame in a video stream individually. However, the free of
charge yet valuable motion information already embedded in the video
compression format is usually overlooked. In this paper, we propose a fast
object detection method by taking advantage of this with a novel Motion aided
Memory Network (MMNet). The MMNet has two major advantages: 1) It significantly
accelerates the procedure of feature extraction for compressed videos. It only
need to run a complete recognition network for I-frames, i.e. a few reference
frames in a video, and it produces the features for the following P frames
(predictive frames) with a light weight memory network, which runs fast; 2)
Unlike existing methods that establish an additional network to model motion of
frames, we take full advantage of both motion vectors and residual errors that
are freely available in video streams. To our best knowledge, the MMNet is the
first work that investigates a deep convolutional detector on compressed
videos. Our method is evaluated on the large-scale ImageNet VID dataset, and
the results show that it is 3x times faster than single image detector R-FCN
and 10x times faster than high-performance detector MANet at a minor accuracy
loss.
