  Predictive learning ideally builds the world model of physical processes in
one or more given environments. Typical setups assume that we can collect data
from all environments at all times. In practice, however, different prediction
tasks may arrive sequentially so that the environments may change persistently
throughout the training procedure. Can we develop predictive learning
algorithms that can deal with more realistic, non-stationary physical
environments? In this paper, we study a new continual learning problem in the
context of video prediction, and observe that most existing methods suffer from
severe catastrophic forgetting in this setup. To tackle this problem, we
propose the continual predictive learning (CPL) approach, which learns a
mixture world model via predictive experience replay and performs test-time
adaptation with non-parametric task inference. We construct two new benchmarks
based on RoboNet and KTH, in which different tasks correspond to different
physical robotic environments or human actions. Our approach is shown to
effectively mitigate forgetting and remarkably outperform the na\"ive
combinations of previous art in video prediction and continual learning.
