  A crucial challenge in reinforcement learning is to reduce the number of
interactions with the environment that an agent requires to master a given
task. Transfer learning proposes to address this issue by re-using knowledge
from previously learned tasks. However, determining which source task qualifies
as the most appropriate for knowledge extraction, as well as the choice
regarding which algorithm components to transfer, represent severe obstacles to
its application in reinforcement learning. The goal of this paper is to address
these issues with modular multi-source transfer learning techniques. The
proposed techniques automatically learn how to extract useful information from
source tasks, regardless of the difference in state-action space and reward
function. We support our claims with extensive and challenging cross-domain
experiments for visual control.
