  The ability to adapt to changes in environmental contingencies is an
important challenge in reinforcement learning. Indeed, transferring previously
acquired knowledge to environments with unseen structural properties can
greatly enhance the flexibility and efficiency by which novel optimal policies
may be constructed. In this work, we study the problem of transfer learning
under changes in the environment dynamics. In this study, we apply causal
reasoning in the offline reinforcement learning setting to transfer a learned
policy to new environments. Specifically, we use the Decision Transformer (DT)
architecture to distill a new policy on the new environment. The DT is trained
on data collected by performing policy rollouts on factual and counterfactual
simulations from the source environment. We show that this mechanism can
bootstrap a successful policy on the target environment while retaining most of
the reward.
