  Recent advances in robot learning have enabled robots to become increasingly
better at mastering a predefined set of tasks. On the other hand, as humans, we
have the ability to learn a growing set of tasks over our lifetime. Continual
robot learning is an emerging research direction with the goal of endowing
robots with this ability. In order to learn new tasks over time, the robot
first needs to infer the task at hand. Task inference, however, has received
little attention in the multi-task learning literature. In this paper, we
propose a novel approach to continual learning of robotic control tasks. Our
approach performs unsupervised learning of behavior embeddings by incrementally
self-organizing demonstrated behaviors. Task inference is made by finding the
nearest behavior embedding to a demonstrated behavior, which is used together
with the environment state as input to a multi-task policy trained with
reinforcement learning to optimize performance over tasks. Unlike previous
approaches, our approach makes no assumptions about task distribution and
requires no task exploration to infer tasks. We evaluate our approach in
experiments with concurrently and sequentially presented tasks and show that it
outperforms other multi-task learning approaches in terms of generalization
performance and convergence speed, particularly in the continual learning
setting.
