  In distributed reinforcement learning, it is common to exchange the
experience memory of each agent and thereby collectively train their local
models. The experience memory, however, contains all the preceding state
observations and their corresponding policies of the host agent, which may
violate the privacy of the agent. To avoid this problem, in this work, we
propose a privacy-preserving distributed reinforcement learning (RL) framework,
termed federated reinforcement distillation (FRD). The key idea is to exchange
a proxy experience memory comprising a pre-arranged set of states and
time-averaged policies, thereby preserving the privacy of actual experiences.
Based on an advantage actor-critic RL architecture, we numerically evaluate the
effectiveness of FRD and investigate how the performance of FRD is affected by
the proxy memory structure and different memory exchanging rules.
