  The ability to transfer in reinforcement learning is key towards building an
agent of general artificial intelligence. In this paper, we consider the
problem of learning to simultaneously transfer across both environments (ENV)
and tasks (TASK), probably more importantly, by learning from only sparse (ENV,
TASK) pairs out of all the possible combinations. We propose a novel
compositional neural network architecture which depicts a meta rule for
composing policies from the environment and task embeddings. Notably, one of
the main challenges is to learn the embeddings jointly with the meta rule. We
further propose new training methods to disentangle the embeddings, making them
both distinctive signatures of the environments and tasks and effective
building blocks for composing the policies. Experiments on GridWorld and Thor,
of which the agent takes as input an egocentric view, show that our approach
gives rise to high success rates on all the (ENV, TASK) pairs after learning
from only 40% of them.
