  Deep Reinforcement Learning (DRL) sometimes needs a large amount of data to
converge in the training procedure and in some cases, each action of the agent
may produce regret. This barrier naturally motivates different data sets or
environment owners to cooperate to share their knowledge and train their agents
more efficiently. However, it raises privacy concerns if we directly merge the
raw data from different owners. To solve this problem, we proposed a new Deep
Neural Network (DNN) architecture with both global NN and local NN, and a
distributed training framework. We allow the global weights to be updated by
all the collaborator agents while the local weights are only updated by the
agent they belong to. In this way, we hope the global weighs can share the
common knowledge among these collaborators while the local NN can keep the
specialized properties and ensure the agent to be compatible with its specific
environment. Experiments show that the framework can efficiently help agents in
the same or similar environments to collaborate in their training process and
gain a higher convergence rate and better performance.
