  General intelligence requires quick adaption across tasks. While existing
reinforcement learning (RL) methods have made progress in generalization, they
typically assume only distribution changes between source and target domains.
In this paper, we explore a wider range of scenarios where both the
distribution and environment spaces may change. For example, in Atari games, we
train agents to generalize to tasks with different levels of mode and
difficulty, where there could be new state or action variables that never
occurred in previous environments. To address this challenging setting, we
introduce a causality-guided self-adaptive representation-based approach,
called CSR, that equips the agent to generalize effectively and efficiently
across a sequence of tasks with evolving dynamics. Specifically, we employ
causal representation learning to characterize the latent causal variables and
world models within the RL system. Such compact causal representations uncover
the structural relationships among variables, enabling the agent to
autonomously determine whether changes in the environment stem from
distribution shifts or variations in space, and to precisely locate these
changes. We then devise a three-step strategy to fine-tune the model under
different scenarios accordingly. Empirical experiments show that CSR
efficiently adapts to the target domains with only a few samples and
outperforms state-of-the-art baselines on a wide range of scenarios, including
our simulated environments, Cartpole, and Atari games.
