  We study the multi-agent safe control problem where agents should avoid
collisions to static obstacles and collisions with each other while reaching
their goals. Our core idea is to learn the multi-agent control policy jointly
with learning the control barrier functions as safety certificates. We propose
a novel joint-learning framework that can be implemented in a decentralized
fashion, with generalization guarantees for certain function classes. Such a
decentralized framework can adapt to an arbitrarily large number of agents.
Building upon this framework, we further improve the scalability by
incorporating neural network architectures that are invariant to the quantity
and permutation of neighboring agents. In addition, we propose a new
spontaneous policy refinement method to further enforce the certificate
condition during testing. We provide extensive experiments to demonstrate that
our method significantly outperforms other leading multi-agent control
approaches in terms of maintaining safety and completing original tasks. Our
approach also shows exceptional generalization capability in that the control
policy can be trained with 8 agents in one scenario, while being used on other
scenarios with up to 1024 agents in complex multi-agent environments and
dynamics.
