  We introduce CAMRL, the first curriculum-based asymmetric multi-task learning
(AMTL) algorithm for dealing with multiple reinforcement learning (RL) tasks
altogether. To mitigate the negative influence of customizing the one-off
training order in curriculum-based AMTL, CAMRL switches its training mode
between parallel single-task RL and asymmetric multi-task RL (MTRL), according
to an indicator regarding the training time, the overall performance, and the
performance gap among tasks. To leverage the multi-sourced prior knowledge
flexibly and to reduce negative transfer in AMTL, we customize a composite loss
with multiple differentiable ranking functions and optimize the loss through
alternating optimization and the Frank-Wolfe algorithm. The uncertainty-based
automatic adjustment of hyper-parameters is also applied to eliminate the need
of laborious hyper-parameter analysis during optimization. By optimizing the
composite loss, CAMRL predicts the next training task and continuously revisits
the transfer matrix and network weights. We have conducted experiments on a
wide range of benchmarks in multi-task RL, covering Gym-minigrid, Meta-world,
Atari video games, vision-based PyBullet tasks, and RLBench, to show the
improvements of CAMRL over the corresponding single-task RL algorithm and
state-of-the-art MTRL algorithms. The code is available at:
https://github.com/huanghanchi/CAMRL
