  Efficient and robust policy transfer remains a key challenge for
reinforcement learning to become viable for real-wold robotics. Policy transfer
through warm initialization, imitation, or interacting over a large set of
agents with randomized instances, have been commonly applied to solve a variety
of Reinforcement Learning tasks. However, this seems far from how skill
transfer happens in the biological world: Humans and animals are able to
quickly adapt the learned behaviors between similar tasks and learn new skills
when presented with new situations. Here we seek to answer the question: Will
learning to combine adaptation and exploration lead to a more efficient
transfer of policies between domains? We introduce a principled mechanism that
can "Adapt-to-Learn", that is adapt the source policy to learn to solve a
target task with significant transition differences and uncertainties. We show
that the presented method learns to seamlessly combine learning from adaptation
and exploration and leads to a robust policy transfer algorithm with
significantly reduced sample complexity in transferring skills between related
tasks.
