  Recently, there have been efforts to improve the performance in sign language
recognition by designing self-supervised learning methods. However, these
methods capture limited information from sign pose data in a frame-wise
learning manner, leading to sub-optimal solutions. To this end, we propose a
simple yet effective self-supervised contrastive learning framework to excavate
rich context via spatial-temporal consistency from two distinct perspectives
and learn instance discriminative representation for sign language recognition.
On one hand, since the semantics of sign language are expressed by the
cooperation of fine-grained hands and coarse-grained trunks, we utilize both
granularity information and encode them into latent spaces. The consistency
between hand and trunk features is constrained to encourage learning consistent
representation of instance samples. On the other hand, inspired by the
complementary property of motion and joint modalities, we first introduce
first-order motion information into sign language modeling. Additionally, we
further bridge the interaction between the embedding spaces of both modalities,
facilitating bidirectional knowledge transfer to enhance sign language
representation. Our method is evaluated with extensive experiments on four
public benchmarks, and achieves new state-of-the-art performance with a notable
margin. The source code is publicly available at
https://github.com/sakura/Code.
