  Modeling multi-agent systems requires understanding how agents interact. Such
systems are often difficult to model because they can involve a variety of
types of interactions that layer together to drive rich social behavioral
dynamics. Here we introduce a method for accurately modeling multi-agent
systems. We present Interaction Modeling with Multiplex Attention (IMMA), a
forward prediction model that uses a multiplex latent graph to represent
multiple independent types of interactions and attention to account for
relations of different strengths. We also introduce Progressive Layer Training,
a training strategy for this architecture. We show that our approach
outperforms state-of-the-art models in trajectory forecasting and relation
inference, spanning three multi-agent scenarios: social navigation, cooperative
task achievement, and team sports. We further demonstrate that our approach can
improve zero-shot generalization and allows us to probe how different
interactions impact agent behavior.
