  Efficient long-short temporal modeling is key for enhancing the performance
of action recognition task. In this paper, we propose a new two-stream action
recognition network, termed as MENet, consisting of a Motion Enhancement (ME)
module and a Video-level Aggregation (VLA) module to achieve long-short
temporal modeling. Specifically, motion representations have been proved
effective in capturing short-term and high-frequency action. However, current
motion representations are calculated from adjacent frames, which may have poor
interpretation and bring useless information (noisy or blank). Thus, for
short-term motions, we design an efficient ME module to enhance the short-term
motions by mingling the motion saliency among neighboring segments. As for
long-term aggregations, VLA is adopted at the top of the appearance branch to
integrate the long-term dependencies across all segments. The two components of
MENet are complementary in temporal modeling. Extensive experiments are
conducted on UCF101 and HMDB51 benchmarks, which verify the effectiveness and
efficiency of our proposed MENet.
