  Reinforcement learning (RL) is widely used in autonomous driving tasks and
training RL models typically involves in a multi-step process: pre-training RL
models on simulators, uploading the pre-trained model to real-life robots, and
fine-tuning the weight parameters on robot vehicles. This sequential process is
extremely time-consuming and more importantly, knowledge from the fine-tuned
model stays local and can not be re-used or leveraged collaboratively. To
tackle this problem, we present an online federated RL transfer process for
real-time knowledge extraction where all the participant agents make
corresponding actions with the knowledge learned by others, even when they are
acting in very different environments. To validate the effectiveness of the
proposed approach, we constructed a real-life collision avoidance system with
Microsoft Airsim simulator and NVIDIA JetsonTX2 car agents, which cooperatively
learn from scratch to avoid collisions in indoor environment with obstacle
objects. We demonstrate that with the proposed framework, the simulator car
agents can transfer knowledge to the RC cars in real-time, with 27% increase in
the average distance with obstacles and 42% decrease in the collision counts.
