  In group activity recognition, the temporal dynamics of the whole activity
can be inferred based on the dynamics of the individual people representing the
activity. We build a deep model to capture these dynamics based on LSTM
(long-short term memory) models. To make use of these ob- servations, we
present a 2-stage deep temporal model for the group activity recognition
problem. In our model, a LSTM model is designed to represent action dynamics of
in- dividual people in a sequence and another LSTM model is designed to
aggregate human-level information for whole activity understanding. We evaluate
our model over two datasets: the collective activity dataset and a new volley-
ball dataset. Experimental results demonstrate that our proposed model improves
group activity recognition perfor- mance with compared to baseline methods.
