  Group activity recognition is a crucial yet challenging problem, whose core
lies in fully exploring spatial-temporal interactions among individuals and
generating reasonable group representations. However, previous methods either
model spatial and temporal information separately, or directly aggregate
individual features to form group features. To address these issues, we propose
a novel group activity recognition network termed GroupFormer. It captures
spatial-temporal contextual information jointly to augment the individual and
group representations effectively with a clustered spatial-temporal
transformer. Specifically, our GroupFormer has three appealing advantages: (1)
A tailor-modified Transformer, Clustered Spatial-Temporal Transformer, is
proposed to enhance the individual representation and group representation. (2)
It models the spatial and temporal dependencies integrally and utilizes
decoders to build the bridge between the spatial and temporal information. (3)
A clustered attention mechanism is utilized to dynamically divide individuals
into multiple clusters for better learning activity-aware semantic
representations. Moreover, experimental results show that the proposed
framework outperforms state-of-the-art methods on the Volleyball dataset and
Collective Activity dataset. Code is available at
https://github.com/xueyee/GroupFormer.
