  Animal pose estimation and tracking (APT) is a fundamental task for detecting
and tracking animal keypoints from a sequence of video frames. Previous
animal-related datasets focus either on animal tracking or single-frame animal
pose estimation, and never on both aspects. The lack of APT datasets hinders
the development and evaluation of video-based animal pose estimation and
tracking methods, limiting real-world applications, e.g., understanding animal
behavior in wildlife conservation. To fill this gap, we make the first step and
propose APT-36K, i.e., the first large-scale benchmark for animal pose
estimation and tracking. Specifically, APT-36K consists of 2,400 video clips
collected and filtered from 30 animal species with 15 frames for each video,
resulting in 36,000 frames in total. After manual annotation and careful
double-check, high-quality keypoint and tracking annotations are provided for
all the animal instances. Based on APT-36K, we benchmark several representative
models on the following three tracks: (1) supervised animal pose estimation on
a single frame under intra- and inter-domain transfer learning settings, (2)
inter-species domain generalization test for unseen animals, and (3) animal
pose estimation with animal tracking. Based on the experimental results, we
gain some empirical insights and show that APT-36K provides a valuable animal
pose estimation and tracking benchmark, offering new challenges and
opportunities for future research. The code and dataset will be made publicly
available at https://github.com/pandorgan/APT-36K.
