  Besides independent learning, human learning process is highly improved by
summarizing what has been learned, communicating it with peers, and
subsequently fusing knowledge from different sources to assist the current
learning goal. This collaborative learning procedure ensures that the knowledge
is shared, continuously refined, and concluded from different perspectives to
construct a more profound understanding. The idea of knowledge transfer has led
to many advances in machine learning and data mining, but significant
challenges remain, especially when it comes to reinforcement learning,
heterogeneous model structures, and different learning tasks. Motivated by
human collaborative learning, in this paper we propose a collaborative deep
reinforcement learning (CDRL) framework that performs adaptive knowledge
transfer among heterogeneous learning agents. Specifically, the proposed CDRL
conducts a novel deep knowledge distillation method to address the
heterogeneity among different learning tasks with a deep alignment network.
Furthermore, we present an efficient collaborative Asynchronous Advantage
Actor-Critic (cA3C) algorithm to incorporate deep knowledge distillation into
the online training of agents, and demonstrate the effectiveness of the CDRL
framework using extensive empirical evaluation on OpenAI gym.
