  We demonstrate the possibility of learning drone swarm controllers that are
zero-shot transferable to real quadrotors via large-scale multi-agent
end-to-end reinforcement learning. We train policies parameterized by neural
networks that are capable of controlling individual drones in a swarm in a
fully decentralized manner. Our policies, trained in simulated environments
with realistic quadrotor physics, demonstrate advanced flocking behaviors,
perform aggressive maneuvers in tight formations while avoiding collisions with
each other, break and re-establish formations to avoid collisions with moving
obstacles, and efficiently coordinate in pursuit-evasion tasks. We analyze, in
simulation, how different model architectures and parameters of the training
regime influence the final performance of neural swarms. We demonstrate the
successful deployment of the model learned in simulation to highly
resource-constrained physical quadrotors performing station keeping and goal
swapping behaviors. Code and video demonstrations are available on the project
website at https://sites.google.com/view/swarm-rl.
