  Convolutional neural networks with spatio-temporal 3D kernels (3D CNNs) have
an ability to directly extract spatio-temporal features from videos for action
recognition. Although the 3D kernels tend to overfit because of a large number
of their parameters, the 3D CNNs are greatly improved by using recent huge
video databases. However, the architecture of 3D CNNs is relatively shallow
against to the success of very deep neural networks in 2D-based CNNs, such as
residual networks (ResNets). In this paper, we propose a 3D CNNs based on
ResNets toward a better action representation. We describe the training
procedure of our 3D ResNets in details. We experimentally evaluate the 3D
ResNets on the ActivityNet and Kinetics datasets. The 3D ResNets trained on the
Kinetics did not suffer from overfitting despite the large number of parameters
of the model, and achieved better performance than relatively shallow networks,
such as C3D. Our code and pretrained models (e.g. Kinetics and ActivityNet) are
publicly available at https://github.com/kenshohara/3D-ResNets.
