  To build a cross-modal latent space between 3D human motion and language,
acquiring large-scale and high-quality human motion data is crucial. However,
unlike the abundance of image data, the scarcity of motion data has limited the
performance of existing motion-language models. To counter this, we introduce
"motion patches", a new representation of motion sequences, and propose using
Vision Transformers (ViT) as motion encoders via transfer learning, aiming to
extract useful knowledge from the image domain and apply it to the motion
domain. These motion patches, created by dividing and sorting skeleton joints
based on body parts in motion sequences, are robust to varying skeleton
structures, and can be regarded as color image patches in ViT. We find that
transfer learning with pre-trained weights of ViT obtained through training
with 2D image data can boost the performance of motion analysis, presenting a
promising direction for addressing the issue of limited motion data. Our
extensive experiments show that the proposed motion patches, used jointly with
ViT, achieve state-of-the-art performance in the benchmarks of text-to-motion
retrieval, and other novel challenging tasks, such as cross-skeleton
recognition, zero-shot motion classification, and human interaction
recognition, which are currently impeded by the lack of data.
