  Video action recognition, which is topical in computer vision and video
analysis, aims to allocate a short video clip to a pre-defined category such as
brushing hair or climbing stairs. Recent works focus on action recognition with
deep neural networks that achieve state-of-the-art results in need of
high-performance platforms. Despite the fast development of mobile computing,
video action recognition on mobile devices has not been fully discussed. In
this paper, we focus on the novel mobile video action recognition task, where
only the computational capabilities of mobile devices are accessible. Instead
of raw videos with huge storage, we choose to extract multiple modalities
(including I-frames, motion vectors, and residuals) directly from compressed
videos. By employing MobileNetV2 as backbone, we propose a novel Temporal
Trilinear Pooling (TTP) module to fuse the multiple modalities for mobile video
action recognition. In addition to motion vectors, we also provide a temporal
fusion method to explicitly induce the temporal context. The efficiency test on
a mobile device indicates that our model can perform mobile video action
recognition at about 40FPS. The comparative results on two benchmarks show that
our model outperforms existing action recognition methods in model size and
time consuming, but with competitive accuracy.
