  Deep 3-dimensional (3D) Convolutional Network (ConvNet) has shown promising
performance on video recognition tasks because of its powerful spatio-temporal
information fusion ability. However, the extremely intensive requirements on
memory access and computing power prohibit it from being used in
resource-constrained scenarios, such as portable and edge devices. So in this
paper, we first propose a two-stage Fully Separable Block (FSB) to
significantly compress the model sizes of 3D ConvNets. Then a feature
enhancement approach named Temporal Residual Gradient (TRG) is developed to
improve the performance of compressed model on video tasks, which provides
higher accuracy, faster convergency and better robustness. Moreover, in order
to further decrease the computing workload, we propose a hybrid Fast Algorithm
(hFA) to drastically reduce the computation complexity of convolutions. These
methods are effectively combined to design a light-weight and efficient ConvNet
for video recognition tasks. Experiments on the popular dataset report 2.3x
compression rate, 3.6x workload reduction, and 6.3% top-1 accuracy gain, over
the state-of-the-art SlowFast model, which is already a highly compact model.
The proposed methods also show good adaptability on traditional 3D ConvNet,
demonstrating 7.4x more compact model, 11.0x less workload, and 3.0% higher
accuracy
