  Real-world dynamical systems often consist of multiple stochastic subsystems
that interact with each other. Modeling and forecasting the behavior of such
dynamics are generally not easy, due to the inherent hardness in understanding
the complicated interactions and evolutions of their constituents. This paper
introduces the relational state-space model (R-SSM), a sequential hierarchical
latent variable model that makes use of graph neural networks (GNNs) to
simulate the joint state transitions of multiple correlated objects. By letting
GNNs cooperate with SSM, R-SSM provides a flexible way to incorporate
relational information into the modeling of multi-object dynamics. We further
suggest augmenting the model with normalizing flows instantiated for
vertex-indexed random variables and propose two auxiliary contrastive
objectives to facilitate the learning. The utility of R-SSM is empirically
evaluated on synthetic and real time-series datasets.
