  In multi-robot systems, achieving coordinated missions remains a significant
challenge due to the coupled nature of coordination behaviors and the lack of
global information for individual robots. To mitigate these challenges, this
paper introduces a novel approach, Bi-level Coordination Learning (Bi-CL), that
leverages a bi-level optimization structure within a centralized training and
decentralized execution paradigm. Our bi-level reformulation decomposes the
original problem into a reinforcement learning level with reduced action space,
and an imitation learning level that gains demonstrations from a global
optimizer. Both levels contribute to improved learning efficiency and
scalability. We note that robots' incomplete information leads to mismatches
between the two levels of learning models. To address this, Bi-CL further
integrates an alignment penalty mechanism, aiming to minimize the discrepancy
between the two levels without degrading their training efficiency. We
introduce a running example to conceptualize the problem formulation and apply
Bi-CL to two variations of this example: route-based and graph-based scenarios.
Simulation results demonstrate that Bi-CL can learn more efficiently and
achieve comparable performance with traditional multi-agent reinforcement
learning baselines for multi-robot coordination.
