  In recent years, reinforcement learning and its multi-agent analogue have
achieved great success in solving various complex control problems. However,
multi-agent reinforcement learning remains challenging both in its theoretical
analysis and empirical design of algorithms, especially for large swarms of
embodied robotic agents where a definitive toolchain remains part of active
research. We use emerging state-of-the-art mean-field control techniques in
order to convert many-agent swarm control into more classical single-agent
control of distributions. This allows profiting from advances in single-agent
reinforcement learning at the cost of assuming weak interaction between agents.
However, the mean-field model is violated by the nature of real systems with
embodied, physically colliding agents. Thus, we combine collision avoidance and
learning of mean-field control into a unified framework for tractably designing
intelligent robotic swarm behavior. On the theoretical side, we provide novel
approximation guarantees for general mean-field control both in continuous
spaces and with collision avoidance. On the practical side, we show that our
approach outperforms multi-agent reinforcement learning and allows for
decentralized open-loop application while avoiding collisions, both in
simulation and real UAV swarms. Overall, we propose a framework for the design
of swarm behavior that is both mathematically well-founded and practically
useful, enabling the solution of otherwise intractable swarm problems.
