  Group Activity Recognition aims to understand collective activities from
videos. Existing solutions primarily rely on the RGB modality, which encounters
challenges such as background variations, occlusions, motion blurs, and
significant computational overhead. Meanwhile, current keypoint-based methods
offer a lightweight and informative representation of human motions but
necessitate accurate individual annotations and specialized interaction
reasoning modules. To address these limitations, we design a panoramic graph
that incorporates multi-person skeletons and objects to encapsulate group
activity, offering an effective alternative to RGB video. This panoramic graph
enables Graph Convolutional Network (GCN) to unify intra-person, inter-person,
and person-object interactive modeling through spatial-temporal graph
convolutions. In practice, we develop a novel pipeline that extracts skeleton
coordinates using pose estimation and tracking algorithms and employ
Multi-person Panoramic GCN (MP-GCN) to predict group activities. Extensive
experiments on Volleyball and NBA datasets demonstrate that the MP-GCN achieves
state-of-the-art performance in both accuracy and efficiency. Notably, our
method outperforms RGB-based approaches by using only estimated 2D keypoints as
input. Code is available at https://github.com/mgiant/MP-GCN
