  Generalization has been one of the major challenges for learning dynamics
models in model-based reinforcement learning. However, previous work on
action-conditioned dynamics prediction focuses on learning the pixel-level
motion and thus does not generalize well to novel environments with different
object layouts. In this paper, we present a novel object-oriented framework,
called object-oriented dynamics predictor (OODP), which decomposes the
environment into objects and predicts the dynamics of objects conditioned on
both actions and object-to-object relations. It is an end-to-end neural network
and can be trained in an unsupervised manner. To enable the generalization
ability of dynamics learning, we design a novel CNN-based relation mechanism
that is class-specific (rather than object-specific) and exploits the locality
principle. Empirical results show that OODP significantly outperforms previous
methods in terms of generalization over novel environments with various object
layouts. OODP is able to learn from very few environments and accurately
predict dynamics in a large number of unseen environments. In addition, OODP
learns semantically and visually interpretable dynamics models.
