  In this paper, we propose a novel approach to predict group activities given
the beginning frames with incomplete activity executions. Existing action
prediction approaches learn to enhance the representation power of the partial
observation. However, for group activity prediction, the relation evolution of
people's activity and their positions over time is an important cue for
predicting group activity. To this end, we propose a sequential relational
anticipation model (SRAM) that summarizes the relational dynamics in the
partial observation and progressively anticipates the group representations
with rich discriminative information. Our model explicitly anticipates both
activity features and positions by two graph auto-encoders, aiming to learn a
discriminative group representation for group activity prediction. Experimental
results on two popularly used datasets demonstrate that our approach
significantly outperforms the state-of-the-art activity prediction methods.
