  Swarm robots offer fascinating opportunities to perform complex tasks beyond
the capabilities of individual machines. Just as a swarm of ants collectively
moves a large object, similar functions can emerge within a group of robots
through individual strategies based on local sensing. However, realizing
collective functions with individually controlled microrobots is particularly
challenging due to their micrometer size, large number of degrees of freedom,
strong thermal noise relative to the propulsion speed, complex physical
coupling between neighboring microrobots, and surface collisions. Here, we
implement Multi-Agent Reinforcement Learning (MARL) to generate a control
strategy for up to 200 microrobots whose motions are individually controlled by
laser spots. During the learning process, we employ so-called counterfactual
rewards that automatically assign credit to the individual microrobots, which
allows for fast and unbiased training. With the help of this efficient reward
scheme, swarm microrobots learn to collectively transport a large cargo object
to an arbitrary position and orientation, similar to ant swarms. We demonstrate
that this flexible and versatile swarm robotic system is robust to variations
in group size, the presence of malfunctioning units, and environmental noise.
Such control strategies can potentially enable complex and automated assembly
of mobile micromachines, programmable drug delivery capsules, and other
advanced lab-on-a-chip applications.
