  In this paper, we tackle the new task of video-based Activated Muscle Group
Estimation (AMGE) aiming at identifying active muscle regions during physical
activity in the wild. To this intent, we provide the MuscleMap dataset
featuring >15K video clips with 135 different activities and 20 labeled muscle
groups. This dataset opens the vistas to multiple video-based applications in
sports and rehabilitation medicine under flexible environment constraints. The
proposed MuscleMap dataset is constructed with YouTube videos, specifically
targeting High-Intensity Interval Training (HIIT) physical exercise in the
wild. To make the AMGE model applicable in real-life situations, it is crucial
to ensure that the model can generalize well to numerous types of physical
activities not present during training and involving new combinations of
activated muscles. To achieve this, our benchmark also covers an evaluation
setting where the model is exposed to activity types excluded from the training
set. Our experiments reveal that the generalizability of existing architectures
adapted for the AMGE task remains a challenge. Therefore, we also propose a new
approach, TransM3E, which employs a multi-modality feature fusion mechanism
between both the video transformer model and the skeleton-based graph
convolution model with novel cross-modal knowledge distillation executed on
multi-classification tokens. The proposed method surpasses all popular video
classification models when dealing with both, previously seen and new types of
physical activities. The database and code can be found at
https://github.com/KPeng9510/MuscleMap.
