  Deep reinforcement learning (DRL) frameworks are increasingly used to solve
high-dimensional continuous control tasks in robotics. However, due to the lack
of sample efficiency, applying DRL for online learning is still practically
infeasible in the robotics domain. One reason is that DRL agents do not
leverage the solution of previous tasks for new tasks. Recent work on
multi-task DRL agents based on successor features (SFs) has proven to be quite
promising in increasing sample efficiency. In this work, we present a new
approach that unifies two prior multi-task RL frameworks, SF-GPI and value
composition, and adapts them to the continuous control domain. We exploit
compositional properties of successor features to compose a policy distribution
from a set of primitives without training any new policy. Lastly, to
demonstrate the multi-tasking mechanism, we present our proof-of-concept
benchmark environments, Pointmass and Pointer, based on IsaacGym, which
facilitates large-scale parallelization to accelerate the experiments. Our
experimental results show that our multi-task agent has single-task performance
on par with soft actor-critic (SAC), and the agent can successfully transfer to
new unseen tasks. We provide our code as open-source at
"https://github.com/robot-perception-group/concurrent_composition" for the
benefit of the community.
