  The goal of continuous sign language recognition(CSLR) research is to apply
CSLR models as a communication tool in real life, and the real-time requirement
of the models is important. In this paper, we address the model real-time
problem through cross-resolution knowledge distillation. In our study, we found
that keeping the frame-level feature scales consistent between the output of
the student network and the teacher network is better than recovering the
frame-level feature sizes for feature distillation. Based on this finding, we
propose a new frame-level feature extractor that keeps the output frame-level
features at the same scale as the output of by the teacher network. We further
combined with the TSCM+2D hybrid convolution proposed in our previous study to
form a new lightweight end-to-end CSLR network-Low resolution input
net(LRINet). It is then used to combine cross-resolution knowledge distillation
and traditional knowledge distillation methods to form a CSLR model based on
cross-resolution knowledge distillation (CRKD). The CRKD uses high-resolution
frames as input to the teacher network for training, locks the weights after
training, and then uses low-resolution frames as input to the student network
LRINet to perform knowledge distillation on frame-level features and
classification features respectively. Experiments on two large-scale continuous
sign language datasets have proved the effectiveness of CRKD. Compared with the
model with high-resolution data as input, the calculation amount, parameter
amount and inference time of the model have been significantly reduced under
the same experimental conditions, while ensuring the accuracy of the model, and
has achieved very competitive results in comparison with other advanced
methods.
