  In this work, we propose Knowledge Integration Networks (referred as KINet)
for video action recognition. KINet is capable of aggregating meaningful
context features which are of great importance to identifying an action, such
as human information and scene context. We design a three-branch architecture
consisting of a main branch for action recognition, and two auxiliary branches
for human parsing and scene recognition which allow the model to encode the
knowledge of human and scene for action recognition. We explore two pre-trained
models as teacher networks to distill the knowledge of human and scene for
training the auxiliary tasks of KINet. Furthermore, we propose a two-level
knowledge encoding mechanism which contains a Cross Branch Integration (CBI)
module for encoding the auxiliary knowledge into medium-level convolutional
features, and an Action Knowledge Graph (AKG) for effectively fusing high-level
context information. This results in an end-to-end trainable framework where
the three tasks can be trained collaboratively, allowing the model to compute
strong context knowledge efficiently. The proposed KINet achieves the
state-of-the-art performance on a large-scale action recognition benchmark
Kinetics-400, with a top-1 accuracy of 77.8%. We further demonstrate that our
KINet has strong capability by transferring the Kinetics-trained model to
UCF-101, where it obtains 97.8% top-1 accuracy.
