  Activity analysis in which multiple people interact across a large space is
challenging due to the interplay of individual actions and collective group
dynamics. We propose an end-to-end approach for learning person trajectory
representations for group activity analysis. The learned representations encode
rich spatio-temporal dependencies and capture useful motion patterns for
recognizing individual events, as well as characteristic group dynamics that
can be used to identify groups from their trajectories alone. We develop our
deep learning approach in the context of team sports, which provide
well-defined sets of events (e.g. pass, shot) and groups of people (teams).
Analysis of events and team formations using NHL hockey and NBA basketball
datasets demonstrate the generality of our approach.
