  Predicting outcomes in sports is important for teams, leagues, bettors,
media, and fans. Given the growing amount of player tracking data, sports
analytics models are increasingly utilizing spatially-derived features built
upon player tracking data. However, player-specific information, such as
location, cannot readily be included as features themselves, since common
modeling techniques rely on vector input. Accordingly, spatially-derived
features are commonly constructed in relation to anchor objects, such as the
distance to a ball or goal, through global feature aggregations, or via
role-assignment schemes, where players are designated a distinct role in the
game. In doing so, we sacrifice inter-player and local relationships in favor
of global ones. To address this issue, we introduce a sport-agnostic
graph-based representation of game states. We then use our proposed graph
representation as input to graph neural networks to predict sports outcomes.
Our approach preserves permutation invariance and allows for flexible player
interaction weights. We demonstrate how our method provides statistically
significant improvements over the state of the art for prediction tasks in both
American football and esports, reducing test set loss by 9% and 20%,
respectively. Additionally, we show how our model can be used to answer "what
if" questions in sports and to visualize relationships between players.
