  While Deep Reinforcement Learning (DRL) has emerged as a promising approach
to many complex tasks, it remains challenging to train a single DRL agent that
is capable of undertaking multiple different continuous control tasks. In this
paper, we present a Knowledge Transfer based Multi-task Deep Reinforcement
Learning framework (KTM-DRL) for continuous control, which enables a single DRL
agent to achieve expert-level performance in multiple different tasks by
learning from task-specific teachers. In KTM-DRL, the multi-task agent first
leverages an offline knowledge transfer algorithm designed particularly for the
actor-critic architecture to quickly learn a control policy from the experience
of task-specific teachers, and then it employs an online learning algorithm to
further improve itself by learning from new online transition samples under the
guidance of those teachers. We perform a comprehensive empirical study with two
commonly-used benchmarks in the MuJoCo continuous control task suite. The
experimental results well justify the effectiveness of KTM-DRL and its
knowledge transfer and online learning algorithms, as well as its superiority
over the state-of-the-art by a large margin.
