  Transfer and adaptation to new unknown environmental dynamics is a key
challenge for reinforcement learning (RL). An even greater challenge is
performing near-optimally in a single attempt at test time, possibly without
access to dense rewards, which is not addressed by current methods that require
multiple experience rollouts for adaptation. To achieve single episode transfer
in a family of environments with related dynamics, we propose a general
algorithm that optimizes a probe and an inference model to rapidly estimate
underlying latent variables of test dynamics, which are then immediately used
as input to a universal control policy. This modular approach enables
integration of state-of-the-art algorithms for variational inference or RL.
Moreover, our approach does not require access to rewards at test time,
allowing it to perform in settings where existing adaptive approaches cannot.
In diverse experimental domains with a single episode test constraint, our
method significantly outperforms existing adaptive approaches and shows
favorable performance against baselines for robust transfer.
