  Animal pose estimation is challenging for existing image-based methods
because of limited training data and large intra- and inter-species variances.
Motivated by the progress of visual-language research, we propose that
pre-trained language models (e.g., CLIP) can facilitate animal pose estimation
by providing rich prior knowledge for describing animal keypoints in text.
However, we found that building effective connections between pre-trained
language models and visual animal keypoints is non-trivial since the gap
between text-based descriptions and keypoint-based visual features about animal
pose can be significant. To address this issue, we introduce a novel
prompt-based Contrastive learning scheme for connecting Language and AniMal
Pose (CLAMP) effectively. The CLAMP attempts to bridge the gap by adapting the
text prompts to the animal keypoints during network training. The adaptation is
decomposed into spatial-aware and feature-aware processes, and two novel
contrastive losses are devised correspondingly. In practice, the CLAMP enables
the first cross-modal animal pose estimation paradigm. Experimental results
show that our method achieves state-of-the-art performance under the
supervised, few-shot, and zero-shot settings, outperforming image-based methods
by a large margin.
