  Action recognition is a vital task in computer vision, and many methods are
developed to push it to the limit. However, current action recognition models
have huge computational costs, which cannot be deployed to real-world tasks on
mobile devices. In this paper, we first illustrate the setting of real-time
action recognition, which is different from current action recognition
inference settings. Under the new inference setting, we investigate
state-of-the-art action recognition models on the Kinetics dataset empirically.
Our results show that designing efficient real-time action recognition models
is different from designing efficient ImageNet models, especially in weight
initialization. We show that pre-trained weights on ImageNet improve the
accuracy under the real-time action recognition setting. Finally, we use the
hand gesture recognition task as a case study to evaluate our compact real-time
action recognition models in real-world applications on mobile phones. Results
show that our action recognition models, being 6x faster and with similar
accuracy as state-of-the-art, can roughly meet the real-time requirements on
mobile devices. To our best knowledge, this is the first paper that deploys
current deep learning action recognition models on mobile devices.
